Using MongoDB Backend Service with Go App in the Swisscom App Cloud

Introduction

In my last post I described how to create and deploy an app, written with the Google Go language, to the Swisscom Application Cloud. This post is a follow up, in which I would like to share some of the learnings, I discovered while extending the Hello World app, following the ideas for the node.js example.

This blog post is not a detailed step by step tutorial, but rather points out the in my opinion important steps to understand how to get a Go app running in the App Cloud with a MongoDB backend. I strongly suggest to read the above linked blog posts as well, as they help to get a better overall understanding.
The full working example could be found in my Github repository.

Like in my last post, the described steps assume to be executed on a Linux system, precisely on Ubuntu.

Add MongoDB as Backend to the App

To be able to locally test the extended version of our app you need a MongoDB installation. On most Linux distributions MongoDB might be installed from the respective repository with the packet manager of choice. In my case, a simple

sudo apt-get install mongodb

does the job. This installs version 2.4.9 of MongoDB. As I found out later, the MongoDB version in use in the Swisscom App Cloud is 3.0.6 (as of writing). Unfortunately the MongoDB client with version 2.4.9 is not compatible with server version 3.0.6, especially if authentication is used. More on this topic later.

On Ubuntu the MongoDB daemon gets started automatically after installation. On other Linux distributions, this may require a manual start.

Connect a Go App to MongoDB

As MongoDB driver package I have chosen gopkg.in/mgo.v2 which you may get with the following command:

go get gopkg.in/mgo.v2

On godoc.org you find the documentation of the mgo.v2 package. The main parts for the usage in our app are the following (only relevant sections, find the complete project on Github).

Insert a new value (while deleting the old):

session, err := mgo.Dial("mongodb://localhost:27017/weatherDB")
if err != nil {
  log.Printf("MongoDB dial err %v\n", err)
}
defer session.Close()

c := session.DB("weatherDB").C("weatherCOLL")

err = c.DropCollection()
if err != nil {
  log.Printf("MongoDB drop collection err %v\n", err)
}

err = c.Insert(Temperature{Temperature: temperature})
if err != nil {
  log.Printf("MongoDB insert err %v\n", err)
}

And querying the saved value:

session, err := mgo.Dial("mongodb://localhost:27017/weatherDB")
if err != nil {
  log.Printf("MongoDB dial err %v\n", err)
}
defer session.Close()

c := session.DB("weatherDB").C("weatherCOLL")

var result Temperature

err = c.Find(nil).One(&result)
if err != nil {
  log.Printf("MongoDB find err %v\n", err)
}

Preparing the App for the Cloud

As you already know from the previous post, in the App Cloud the configuration (e. g. port to bind, connection details for the backend services) are provided by environment variables. In the “Hello World” example from the first post, the only information, we needed to get from the environment was the port to bind. In this example we used the os package from the Go standard library to get the environment variable. The connection details for the MongoDB instance in the App Cloud will also be provided as environment variable, where the value is a JSON document. It would be possible to parse this JSON document with the encoding/json package from the Go standard library, but luckily there is even an easier solution. Cloud Foundry provides the go-cfenv (godoc.org) Go package, which

“… provides information about the current app deployed on Cloud Foundry, including any bound service(s).”

This is exactly what we need. Get the package as usual with:

go get github.com/cloudfoundry-community/go-cfenv

In the main function of our Go program we add:

// Get settings from ENV if present (e. g. if in App Cloud)
appEnv, err := cfenv.Current()
if err == nil {
  // Port to bind web app
  port = strconv.Itoa(appEnv.Port)

  name = appEnv.Name
  log.SetPrefix(name + prefixDelim)

  // MongoDB Service
  mgoService, err := appEnv.Services.WithName("mongodb")
  if err == nil {
    var ok bool
    mongouri, ok = mgoService.Credentials["uri"].(string)
    if !ok {
      log.Fatalf("No valid MongoDB uri\n")
    }
    mongodb, ok = mgoService.Credentials["database"].(string)
    if !ok {
      log.Fatalf("No valid MongoDB database\n")
    }
  }
}

The above code snipped processes all the relevant environment variables and puts them in Go structs, where we can access them easily.

Adding a MongoDB Service

The steps, required to add MongoDB service to your App Cloud space are quite easy. You have the possibility to add the service in the Developer Console or you may use the command line. This assumes you already installed the Cloud Foundry CLI tools and you logged in to the App Cloud with the CLI tools (see previous blog post).

cf create-service mongodb small mongodb

If your app is already existing in the App Cloud, for example by pushing the “Hello World” example, you may now bind the MongoDB service to your app:

cf bind-service DemoApp mongodb

Update our Go Dependencies with Godep

Before we can push our updated app to the App Cloud, we have to update our dependencies with the godep tool, by running:

godep save -r

This will place the newly introduced Go packages (gopkg.in/mgo.v2 and github.com/cloudfoundry-community/go-cfenv) including their dependencies into the Godeps folder and update the import statements accordingly.

Push the Updated App to the Cloud

With the MongoDB service created, the binding between app and MongoDB service in place, the app prepared to read the configuration from the environment variables and our Go package dependencies updated, we are ready to push our updated app to the cloud by simply executing cf push. You may now access the updated app with your browser and enjoy playing with the weather.

Directly Connect to MongoDB Service in the App Cloud

The Swisscom App Cloud allows to setup a “network tunnel” from your local machine to the backend services in the App Cloud. This requires the Service Connector plugin for the Cloud Foundry CLI tools. The installation is simple, to get the Linux 64 bit version of the plugin just execute:

cf install-plugin https://swisscom-plugin.scapp.io/linux64/swisscom-plugin

The list of the available pre-built plugins for the different platforms (Windows, OSX, 32 or 64 bit) is available in the chapter Managing Services in the App Cloud documentation.

Before we can open the network tunnel to the App Cloud, we have to create a so called service key on the respective service. This may be done with the following command, where “local” is the name of the newly created service key:

cf create-service-key mongodb local

To get the just created service key credentials execute:

cf service-key mongodb local

The above command prints something similar to:

{
  "database": "mxAa4xiVidUHrO48",
  "database_uri": "mongodb://PJq5jZbdp6Bqshmn:FSCEvxfMPgd62Q1Z@uncvb9blaeo7t3mw.service.consul:59629/mxAa4xiVidUHrO48",
  "host": "uncvb9blaeo7t3mw.service.consul",
  "password": "FSCEvxfMPgd62Q1Z",
  "port": "59629",
  "uri": "mongodb://PJq5jZbdp6Bqshmn:FSCEvxfMPgd62Q1Z@uncvb9blaeo7t3mw.service.consul:59629/mxAa4xiVidUHrO48",
  "username": "PJq5jZbdp6Bqshmn"
}

With the credentials at hand, we may now setup the network tunnel by executing (replace [host] and [port] according to the generated credentials in the service key):

cf service-connector 13000 [host]:[port]

The above command will not exit until receiving SIGINT, normally sent by Ctrl+C. While running, it will listen on the provided port, in the above example port 13000. We now can connect with our local MongoDB client to this local port, which will forward our connection to the MongoDB instance in the App Cloud (replace [username], [database] and [password] according to the generated credentials in the service key):

mongo -username [username] -host localhost -port 13000 [database] -password [password]

The password may be omitted in the above command, in this case the password will be prompted, which prevents from leaking sensitive information to the process list or the shell history.

As mentioned in the section about the MongoDB installation, the above command didn’t work for me in the first place. It always throwed the following error:

MongoDB shell version: 2.4.9
connecting to: localhost:13000/mxAa4xiVidUHrO48
Mon Dec 21 22:06:39.495 Error: 18 { ok: 0.0, errmsg: "auth failed", code: 18 } at src/mongo/shell/db.js:228
exception: login failed

This error message didn’t disappear until I updated my local MongoDB installation to a more recent version. For this update I followed the instructions in the Install MongoDB on Ubuntu tutorial.