Archive for the ‘Cloud Foundry’ Category

TCP Routing and SSL: A Walkthrough Using Spring Boot An incredible guest blog by Ben Dalby, Advisory Consultant at DellEMC

An incredible guest blog by Ben Dalby, Advisory Consultant at DellEMC

Walkthrough: Cloud Foundry TCP Routing and SSL

Guest Blog by Ben Dalby, Advisory Consultant (Applications and Big Data) at DellEMC

Use Cloud Foundry’s TCP routing feature to terminate SSL directly in your application

Introduction

A common security requirement for customers in regulated industries such as banking and healthcare is that all traffic should be secured end-to-end with SSL.

Prior to Pivotal Cloud Foundry 1.8, inbound SSL connections would always terminate on the Gorouter, and further encryption could only be achieved between the Gorouter and running applications by installing Pivotal’s IPsec Add-on

With the introduction in version 1.8 of TCP routing, it is now possible to terminate SSL right at your application – and this article will walk you through a working example of a Spring Boot application that is secured with SSL in this way.

Prerequisites

PCF Dev version 0.23.0 or later
JDK 1.8 or later
Gradle 2.3+ or Maven 3.0+
git (tested on 2.10.1)
A Linux-like environment (you will need to change the file paths for the directory commands to work on Windows)

How to do it

Step 1 – Create a Spring Boot application

We’re going to be lazy here, and simply make a couple of small modifications to the Spring Boot Getting Started application:

$ git clone https://github.com/spring-guides/gs-spring-boot.git

Step 2 – Create an SSL certificate

$ cd [GITHUB HOME]/gs-spring-boot/intial/src/main/resources
$ keytool -genkey -alias tomcat -storetype PKCS12 -keyalg RSA -keysize 2048 \
-keystore keystore.p12 -validity 3650 -keypass CHANGEME -storepass CHANGEME \
-dname "C=GB,ST=Greater London,L=London,O=Dell EMC,OU=Apps and Data,CN=abd.dell.com"  

Step 3 – Configure Spring Boot to use SSL and the new certificate

(You can also retrieve the application.properties shown below from here)

$ cd [GITHUB HOME]/gs-spring-boot/intial/src/main/resources
$ cat <<EOT >> application.properties  
server.port: 8080
server.ssl.key-store: classpath:keystore.p12
server.ssl.key-store-password: CHANGEME
server.ssl.keyStoreType: PKCS12
server.ssl.keyAlias: tomcat
EOT  

Step 4 – Package the application

$ cd [GITHUB HOME]/gs-spring-boot/intial  
$ mvn clean package  

Step 5 – Push the application to PCF Dev (use default org and space)

$ cd [GITHUB HOME]/gs-spring-boot/intial
$ cf target -o pcfdev-org -s pcfdev-space
$ cf push gs-spring-boot -p target/gs-spring-boot-0.1.0.jar  

Step 6 – Create a TCP route and map it to your application

$ cf create-route pcfdev-space tcp.local.pcfdev.io --random-port
Creating route tcp.local.pcfdev.io for org pcfdev-org / space pcfdev-space as admin...
OK
Route tcp.local.pcfdev.io:61015 has been created
$ cf map-route gs-spring tcp.local.pcfdev.io --port 61015

Step 7 – Verify you can now connect directly to your application over SSL

Browse to https://tcp.local.pcfdev.io:61015/ (substitute your own port after the colon):

View details of the certificate to verify it is the one you just generated (note the procedure has just changed if you are using Chrome):

Further Reading

Enabling TCP Routing
http://docs.pivotal.io/pivotalcf/1-9/adminguide/enabling-tcp-routing.html

How to tell application containers (running Java apps) to trust self-signed certs or a private/internal CA https://discuss.pivotal.io/hc/en-us/articles/223454928-How-to-tell-application-containers-running-Java-apps-to-trust-self-signed-certs-or-a-private-internal-CA

Enable HTTPS in Spring Boot
https://drissamri.be/blog/java/enable-https-in-spring-boot/

Do You Hear That? It’s the sound of Keyboards! Call for Papers | Cloud Foundry Summit Silicon Valley 2017 is quickly approaching!

Call for Papers | Cloud Foundry Summit Silicon Valley 2017 is quickly approaching!

Our brains are on fire, our keyboards are hot, and the joke in the office the past few days has been over our extreme excitement for the eventual need to buy sunscreen since our Boston winter leaves us Vitamin D deprived. Why is this the case, you may or may not be asking? Well, I plan on telling you anyway because it is just too exciting not to share!

Our team is preparing for CLOUD FOUNDRY SUMMIT SILICON VALLEY! We felt a social duty to let all of those we care about and want to be there with us for what’s sure to be the summit of the summer (how can it not be when it is being held in June in Santa Clara?!), that the last call for papers is quickly approaching (no seriously, it’s this Friday, February 17th).

Just as a refresher for those on the fence, Cloud Foundry Summit is the premier event for enterprise app developers. This year the Foundation, through market research and feedback, found that interest and industry need is engrained in the focus on innovation and the streamlining of development pipelines. For this reason, Summit 2017 is majorly honing in on microservices and continuous delivery in developers’ language and framework of choice. That is why the session tracks available will be Use Cases, Core Project Updates, Experiments, Extension Projects, and Cloud Native Java. Each session that is chosen for the conference is enabled (1) primary speaker and (1) co-speaker. The primary speaker receives a complimentary conference pass while the co-speaker receives a discounted conference pass. So what’s stopping us from getting involved? Absolutely NOTHING!

As a sneak peak to a few of the topics our team have submitted for approval, see below:

  • Adopting DevOps and Building a Cloud Foundry Dojo (Lessons Learned)
  • Lift & Shift Your Legacy Apps to Cloud Foundry
  • How to Develop Scalable Cloud Native Application with Cloud Foundry
  • Enabling GPU-as-a-Service in Cloud Foundry
  • Blockchain as a Service
  • Avoiding pitfalls while migrating BOSH deployments
  • Spring Content: Cloud-Native Content Services for Spring

 

So, now what’s stopping YOU from getting involved? Submit papers here: https://www.cloudfoundry.org/cfp-2017/ and/or register here: https://www.regonline.com/registration/Checkin.aspx?EventID=1908081&utm_source=flash&utm_campaign=summit_2017_sv&utm_medium=landing&utm_term=cloud%20foundry%20summit&_ga=1.199163247.1732851993.1460056335

Last but definitely not least, let us know if you plan on coming—we are more than happy to share sunscreen 🙂 We cannot wait to see you there!

Getting Started with Cloud Foundry Volume Services Trying out Persistent File Systems in Your Cloud Foundry Applications with PCFDev

Trying out Persistent File Systems in Your Cloud Foundry Applications with PCFDev

Julian Hjortshoj

Julian Hjortshoj

Julian is a 12 year veteran of DellEmc, and the current PM of the Cloud Foundry Diego Persistence team.

This week, the PCFDev team released a new version of PCFDev that includes local-volume services out of the box.  This new release of PCFDev gives us an option to get up and running with volume services that is an order of magnitude easier than any of the options we had before. We thought it would be a good time to write a post detailing the steps to try out volume services with your Cloud Foundry applications, now that the post doesn’t need to be quite so long.

1. Install PCFDev

(Time: 30-90 minutes, depending on your internet connection speed, and what you already have installed.)

Instructions for PCFDev installation can be found here.  Installation requires VirtualBox and the Cloud Foundry CLI, and you will need to have (or create) an account on Pivotal Network.  Don’t be deterred.  It’s free and easy.  If you already have PCFDev, make sure that you have version 0.22.0 or later.

Once you have installed PCFDev and successfully started it up, you should see something like this:

screen-shot-2016-11-16-at-4-39-13-pm

Now log in and select the pcfdev-org space:

○ → cf login -a https://api.local.pcfdev.io --skip-ssl-validation
API endpoint: https://api.local.pcfdev.io

Email> admin

Password>
Authenticating...
OK

Select an org (or press enter to skip):
1. pcfdev-org
2. system

Org> 1
Targeted org pcfdev-org

Targeted space pcfdev-space

API endpoint: https://api.local.pcfdev.io (API version: 2.58.0)
User: admin
Org: pcfdev-org
Space: pcfdev-space

○ →

2. Create a New Volume

(Time: 1 minute)

If you have the right version of PCFDev, then the local-volume service broker should already be installed.  You can tell this by typing

cf marketplace

You should see local-volume in the list of available services:

screen-shot-2016-11-16-at-4-43-59-pm

Use cf create-service to create a new file volume that you can bind to your application:

○ → cf create-service local-volume free-local-disk myvolume
Creating service instance myvolume in org pcfdev-org / space pcfdev-space as admin...
OK

○ →

3. Push an Application

(Time: 5 minutes)

For the purposes of this blog post, we’ll use the “pora” test application that we use in our persi CI pipeline. “Pora” is the persistent version of the Cloud Foundry acceptance test “Dora” application.  It is a simple ‘hello world” app that writes a message into a file, and then reads it back out again.

To get the Pora app, first clone the persi-acceptance-tests github repo:

○ → git clone https://github.com/cloudfoundry-incubator/persi-acceptance-tests
Cloning into 'persi-acceptance-tests'...
remote: Counting objects: 228, done.
remote: Compressing objects: 100% (97/97), done.
remote: Total 228 (delta 120), reused 222 (delta 114), pack-reused 0
Receiving objects: 100% (228/228), 39.42 KiB | 0 bytes/s, done.
Resolving deltas: 100% (120/120), done.
Checking connectivity... done.

○ → 

Now change to the pora folder, and push the app to cf with the “no-start” option:

○ → cd persi-acceptance-tests/assets/pora

○ → cf push pora --no-start
Using manifest file /Users/pivotal/workspace/eraseme/persi-acceptance-tests/assets/pora/manifest.yml

Creating app pora in org pcfdev-org / space pcfdev-space as admin...
OK

Creating route pora.local.pcfdev.io...
OK

Binding pora.local.pcfdev.io to pora...
OK

Uploading pora...
Uploading app files from: /Users/pivotal/workspace/eraseme/persi-acceptance-tests/assets/pora
Uploading 1K, 2 files
Done uploading
OK
○ →

4. Bind the Service to the Application and Start the App

(Time: 5 minutes)

The cf “bind-service” command makes our new volume available to the Pora application.

○ → cf bind-service pora myvolume
Binding service myvolume to app pora in org pcfdev-org / space pcfdev-space as admin...
OK
TIP: Use 'cf restage pora' to ensure your env variable changes take effect

○ →

Now start the pora application:

○ → cf start pora
Starting app pora in org pcfdev-org / space pcfdev-space as admin...
Downloading go_buildpack...
Downloaded go_buildpack (320.8M)
Creating container
Successfully created container
Downloading app package...
Downloaded app package (1.1K)
Staging...
-------> Buildpack version 1.7.10
file:///tmp/buildpacks/a62bb8a9fc6ecb01a06f86c020c7a142/dependencies/https___buildpacks.cloudfoundry.org_concourse-binaries_godep_godep-v74-linux-x64.tgz
file:///tmp/buildpacks/a62bb8a9fc6ecb01a06f86c020c7a142/dependencies/https___buildpacks.cloudfoundry.org_concourse-binaries_glide_glide-v0.11.0-linux-x64.tgz
-----> Installing go1.6.3... done
Downloaded [file:///tmp/buildpacks/a62bb8a9fc6ecb01a06f86c020c7a142/dependencies/https___storage.googleapis.com_golang_go1.6.3.linux-amd64.tar.gz]
-----> Running: go install -v -tags cloudfoundry .
pora
Exit status 0
Staging complete
Uploading droplet, build artifacts cache...
Uploading build artifacts cache...
Uploading droplet...
Uploaded build artifacts cache (81.6M)
Uploaded droplet (2.3M)
Uploading complete
Destroying container
Successfully destroyed container

1 of 1 instances running

App started

OK

App pora was started using this command `pora`

Showing health and status for app pora in org pcfdev-org / space pcfdev-space as admin...
OK

requested state: started
instances: 1/1
usage: 256M x 1 instances
urls: pora.local.pcfdev.io
last uploaded: Thu Nov 17 03:20:29 UTC 2016
stack: cflinuxfs2
buildpack: go_buildpack

state since cpu memory disk details
#0 running 2016-11-16 07:38:32 PM 0.0% 0 of 256M 0 of 512M

○ →

Once the app is started, you can use curl with the reported url to make sure it is reachable.  The default endpoint for the application will just report back the instance index for the application:

○ → curl pora.local.pcfdev.io
instance index: 0
○ →

To test the persistence feature, add “/write” to the end of the url.  This will cause the pora app to pull the location of the shared volume from the VCAP_SERVICES environment variable, create a file, write a message into it, and then read the message back out:

○ → curl pora.local.pcfdev.io/write
Hello Persistent World!
○ →

5. Use Persistent Volumes with Your Own Application

By default, the volume service broker will generate a container mount path for your mounted volume, and it will pass that path into your application via the VCAP_SERVICES environment variable. You can see this when you type “cf env pora” which produces output like this:

{
 "VCAP_SERVICES": {
  "local-volume": [
   {
    ...
    "volume_mounts": [
     {
      "container_path": "/var/vcap/data/0ce3b464-3d4c-45af-9e78-29990d7ddac1",
      "mode": "rw"
     }
    ]
   }
  ]
 }
}

In your application, you can parse the VCAP_SERVICES environment variable as json and determine the value of “container_path” to find the location of your shared volume in the container, or simply parse out the path with a regular expression.  You can see an example of this code flow in the Pora application here.

If it isn’t practical to connect to an arbitrary location in your application for some reason, (e.g. the directory is hard coded everywhere, or your code is precompiled, or you are reluctant to change it) then you can also tell the broker to put the volume in a specific location when you bind the application to the service.  To do that, unbind the service, and then rebind it using the -c option.  This will create any necessary directories on the container, and put the volume mount in the specific path you need:

○ → cf unbind-service pora myvolume
Unbinding app pora from service myvolume in org pcfdev-org / space pcfdev-space as admin...
OK
○ → cf bind-service pora myvolume -c '{"mount":"/my/specific/path"}'
Binding service myvolume to app pora in org pcfdev-org / space pcfdev-space as admin...
OK
TIP: Use 'cf restage pora' to ensure your env variable changes take effect
○ →

Type “cf restage pora” to restart the application in this new configuration, and you should observe that the application operates as before, except that it now accesses the shared volume from “/my/specific/path”.  You can also see the new path transmitted to the application by typing “cf env pora”.

6. If You Need to Copy Data

If you need to copy existing data into your new volume in order for your application to use it, you will need to get the data onto the PCFDev host, and then copy it into the source directory that the local-volume service uses to mimic a shared volume.  In a real-world scenario, these steps wouldn’t be necessary because the volume would come from a real shared filesystem that you could mount and copy content to, but the local-volume service is a test service that mimics shared volumes for simple cloudfoundry deployments.

The key that PCFDev uses for scp/ssh is located in ~/.pcfdev/vms/key.pem.  This file must be tweaked to reduce its permissions before it can be used by scp:

○ → chmod 0600 ~/.pcfdev/vms/key.pem
○ →

Now invoke scp to copy your content across to the PCFDev VM.  The example below copies a file named “assets.tar” to the var/vcap/data directory on the VM:

○ → scp -i ~/.pcfdev/vms/key.pem ./assets.tar vcap@local.pcfdev.io:/var/vcap/data
assets.tar 100% 10KB 10.0KB/s 00:00
○ →

Now, use “cf dev ssh” to open a ssh session into the vm, and cd to /var/vcap/data.  You should find your file there:

○ → cf dev ssh
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.19.0-74-generic x86_64)

* Documentation: https://help.ubuntu.com/
New release '16.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

vcap@agent-id-pcfdev-0:~$ cd /var/vcap.data
-bash: cd: /var/vcap.data: No such file or directory
vcap@agent-id-pcfdev-0:~$ cd /var/vcap/data
vcap@agent-id-pcfdev-0:/var/vcap/data$ ls
assets.tar cloud_controller_ng executor_cache jobs nginx_cc stager tmp uaa voldrivers
bbs executor garden mysql packages sys tps upgrade_preparation_nodes volumes
vcap@agent-id-pcfdev-0:/var/vcap/data$

“ls volumes/local/_volumes” gives us a listing of the volumes that have been created.  If you are following this tutorial, there should be only one directory in there, with a long ugly name.  Move the file into that directory, and then exit the ssh session

vcap@agent-id-pcfdev-0:/var/vcap/data$ ls volumes/local/_volumes/
843f99cc-7638-4a0f-8209-b522928cd443
vcap@agent-id-pcfdev-0:/var/vcap/data$ mv assets.tar volumes/local/_volumes/843f99cc-7638-4a0f-8209-b522928cd443/
vcap@agent-id-pcfdev-0:/var/vcap/data$ exit
logout
○ →

Finally, invoke “cf ssh pora” to open a ssh session into the cloudfoundry app.  This will allow you to verify that the data is available to your application.  If you used the -c option to specify a mount path, you should find your content there.  If not, you will find it under “/var/vcap/data/{some-guid}”:

○ → cf ssh pora
vcap@2bts9kfetl8:~$ ls /my/specific/path/
assets.tar test.txt
vcap@2bts9kfetl8:~$

7. Where to Find More Information

The best place to reach the CF Diego Persistence team is on our Slack channel here: https://cloudfoundry.slack.com/messages/persi/. We’re happy to hear from you with any questions you have, and to help steer you in the right direction.

If you don’t already have an account on CF slack, you can create an account here: https://cloudfoundry.slack.com/signup

For an introduction to Volume Services in Cloud Foundry, refer to this document: http://bit.ly/2gHPVJM

Or watch our various presentations this year:

CF Summit Frankfurt 2016: https://www.youtube.com/watch?v=zrQPns47xho

Spring One Platform 2016: https://www.youtube.com/watch?v=VisP5ebZoWw

CF Summit Santa Clara 2016: https://www.youtube.com/watch?v=ajNoPi1uMjQ

Bringing Cloud Foundry to You… Exciting Upcoming Events that the Cool Kids are Attending

Exciting Upcoming Events that the Cool Kids are Attending

For those of you who follow us on Twitter (@DellEMCDojo for those that don’t) or via this blog, I would guess it is pretty evident that we just love Cloud Foundry. We take pride in being a member of the foundation mostly because it is with this platform that we feel we are delivering and practicing what we preach; living out a true DevOps culture most conducive to productivity and quality product and delivery.

We are lucky enough to just be returning from Cloud Foundry Summit where upwards of 800 developers and other market adopters joined in Frankfurt, Germany to learn from other community members and to be a part of the PaaS’s business direction. With the Native Hybrid Cloud team, we at the Dojo spent our days representing DellEMC through presentations and booth shifts. We left feeling even more confident in our daily choice to use Cloud Foundry and to further evangelize the PaaS for the betterment of the larger technical OS community.

It is because of this that we are excited to announce that we will be hosting and supporting three Cloud Foundry Days here in the near future. Cloud Foundry Days are new, one-day educational events aimed at developers and operators who want to connect with businesses, learn about the platform, and share knowledge. The first that we will be hosting with the support of TIBCO, Pivotal, Telstra, Stark & Wayne, among others, is going to be taking place on October 18th in Sydney, Australia. If you have any interest in attending or know of others who may be, please find the registration link here: https://www.eventbrite.com.au/e/cloud-foundry-day-sydney-australia-tickets-27935602138. The second of these events will be taking place on October 20th in Melbourne, Australia and will also be supported by TIBCO, Pivotal, Telstra, Stark & Wayne, among others. The registration link for this event can be found here: https://www.eventbrite.com.au/e/cloud-foundry-day-melbourne-australia-tickets-27985249635. Last, but definitely not least, is the CF Day in Shanghai, China on October 25th where we will be supporting GE (Registration information to be available within the next day or two. Please contact us via twitter if you are interested).

We truly hope that these events will be fruitful in their reach and effectiveness, and help our technical community discern the best option for themselves and the businesses they affect. Teamwork makes the dream work!

NFS app support in under 5 minutes with CloudFoundry, Wahooo!

Peter Blum

Developers want quick and easy access to storage! They don’t care how they get it, they just want it now. Let me show you how a developer can go from 0 to 100 with persistence in CloudFoundry, starting with Dell EMC Isilon.

Cloud Foundry. Open Source. The Way. Dell EMC [⛩] Dojo.

Oh look! It’s a Docker Container Running on CloudFoundry!

Peter Blum

Not many people know how simple it is to run and scale there containerized application on CloudFoundry. Checkout this simple and cool video showcasing how it can be done!

Cloud Foundry. Open Source. The Way. Dell EMC [⛩] Dojo.

 

A Fresh Developer’s take on Cloud Foundry Why the third platform is the new way to iterate

Why the third platform is the new way to iterate

Coming to the Dojo a little over two months ago for my first day of work at my first (out of college) job, I was mystified by the amount of understanding and knowledge I still had to gain by using the Cloud Platform as a development and deployment environment. After the first few weeks, we started developing a brand new application for internal use. There was no existing infrastructure, but a clear idea to use a SQL database, and a web portal for the UI. I was thrilled! I worked on my fair share of web applications, so this would be a cake walk.

Great! What were the IP’s? What was the endpoint? What was the downtime for our environment on updates? What was our server technology? They said it was up to me. I began thinking of asking IT for a VM spin up our database, scripts designed to back up the data with cron jobs, registering a DNS for our website, and for a potential duplicate set for our test environment; technical aches and pains that are first steps when building a web application. I then began asking what seemed the obvious question of who to go to for the resources, and I was told we would do it all ourselves. Wait, I thought, we were supposed to deploy and maintain the VM’s set up users for the database, download the applications, and backup technology? That was easily another two or three days short term, and days of complications long term. Or so I thought.

Except…I hadn’t known that we were using Cloud Foundry, and the benefits that came from this. All of the pain points I used to face in setting up a functioning web app were moot. Instead of spending time configuring and maintaining environments, the Dojo, as a team, has put together a full-functioning web application with production environment, architecture, an extensive automated testing harness, and automated deployment. All as painlessly as we might use Github.

Removing Development Roadblocks

When I thought of the Web application, a typical structure comes to mind of the architecture for the app.

Screen Shot 2016-08-31 at 12.20.49 PM

A typical developer headache for a web application

There’s plenty of steps for our team, as developers, to maintain and care about here. But with Cloud Foundry, we can remove many of these concerns.

Lets talk about Databases first. Using service brokers as provided through CF, we can simply bind a SQL service to our application. For extra sweetness, we can use a buildpack that supports that our server is connecting to a service-provided database. The buildpack will, at runtime, connect our application’s backend into a bound SQL database, no environment variables necessary.

Servers and UI? No problem. Pushing applications to Cloud Foundry takes as many commands as pushing to git. We can also update and configure the application directly through a UI included in our environment. This means changing something we forgot to put in our three-four deployment commands doesn’t warrant another deployment. As far as firewalls go, using the proper security groups to avoid exposing secretive endpoints is a few minutes behind the scenes of a UI, or a couple commands in a terminal.

Alright, so our development roadblocks are covered. What does all this mean for our Customers?

Rapid Integration and Feedback Loops

With our Test Driven Development and Continuous Integration mantras, we can rapidly iterate on our products. With Cloud Foundry, those rapid iterations allow us to take customer feedback. Using our real product on something as quick to set up and easily accessible as CF platform is, we can respond to our customer’s concerns much more efficiently.

Allowing our customers a way to directly interact with our product in this way is not only helpful for them, but for us, for a variety of reasons. Firstly, we’re able to get valuable feedback from them in a timely fashion, not when a feature has been done and untouched for two months (less old code touching, the better). Additionally, we can test what our real runtime environment will be like for our production level product, which means faster satisfaction for customers.

Screen Shot 2016-08-31 at 12.48.14 PM

Our development cycle. Made simple with less setup behind the scenes

Finally, we don’t have to worry about maintaining servers and VMS when they may go down. Cloud Foundry can sense when an app crashes, and restart it for us. Same database, same DNS, and limited downtime. Doesn’t get much easier than that.

Final thoughts

Using Cloud Foundry also means we can simplify our development process by having a consistent API and well-documented environment surrounding our application. When we rotate around our projects, people coming into our web application are using similar knowledge and practices they might use when deploying our Minecraft Docker Containers.

It’s for the reasons in this article, and I’m sure for many more to come, that the cloud platform and Cloud Foundry has made development easier for us. It allows us to create better products, faster. We can get immediate feedback from customers and get stories accepted faster. For most developers, that’s a no brainer.

Multi-Cloud MineCraft! Minecraft + Docker + CloudFoundry + vSphere + RackHD + ScaleIO = Awesomeness

Minecraft + Docker + CloudFoundry + vSphere + RackHD + ScaleIO = Awesomeness

Peter Blum

If you didn’t know, VMworld begins next week! Here at the Dojo we thought we would show you a cool project we’ve been working on. MINECRAFT in the cloud!!

With all the clouds out there, both public and private, we wanted to bridge the gap between them. We took a public vSphere cloud and a private bare metal cloud, and layered CloudFoundry ontop. Then with a simple CF push we spin up a Minecraft server from a Docker image. Hmm don’t know if I could put more buzz words into a sentence if I tried…anyhow check it out!

Big shout out to VMWare, RackHD, RexRay, Libstorage, and of course CloudFoundry for helping us make it possible. Also here is a link to our Github (https://github.com/EMC-Dojo/CloudFoundry-MineCraft) where you can find the Docker Image that you can run inside cloudFoundry for your own private Minecraft server.

Cloud Foundry. Open Source. The Way. EMC [⛩] Dojo.

Using Docker Container in Cloud Foundry

Using Docker Container in Cloud Foundry

As we all know, we can push source code to CF directly, and CF will compile it and create a container to run our application. Life is so great with CF.

But sometimes, for some reason, such as our App needs a special setup or we want to run an app on different platforms or infrastructures, we may already have a preconfigured container for our App. This won’t block our way to CF at all. This post will show you how to push docker images to CF.

Enable docker feature for CF

We can turn on docker support with the following cf command

  cf enable-feature-flag diego_docker

We can also turn it off by

  cf disable-feature-flag diego_docker
Push docker image to CF
  cf push cf-docker -o golang/alpine

Unlike the normal way, CF won’t try to build our code and run it inside the image we specified. CF would assume that you already put  everything you need into your docker image. We have to rebuild the docker image every time we push a change to our repository.

We also need to tell CF how to start our app inside the image by specifying the start command. We can either put it as an argument for cf push or put it into manifest.yml as below.

---
applications:
- name: cf-docker
  command: git clone https://github.com/kaleo211/cf-docker && cd cf-docker && mkdir -p app/tmp && go run main.go

In this example, we are using an official docker image from docker hub. In the start command, we clone our demo repo from Github, do something and run our code.

Update Diego with private docker registry

If you are in the EMC network, you may not able to use Docker Hub due to certificate issues. In this case, you need to setup a private docker registry. The version of registry needs to be V2 for now. Also, you have to redeploy your CF or Diego with the changes being shown below.

properties:
  garden:
    insecure_docker_registry_list:
    - 12.34.56.78:9000
  capi:
    stager:
      insecure_docker_registry_list:
      - 12.34.56.78:9000

Replace 12.34.56.78:9000 with your own docker registry ip and port.

Then, you need to create a security group to reach your private docker registry. You can put the definition of this security group into docker.json as shown below

[
    {
        "destination": "12.34.56.78:9000",
        "protocol": "all"
    }
]

And run

  cf create-security-group docker docker.json
  cf bind-staging-security-group docker

Now you can re-push to CF by

  cf push -o 12.34.56.78:9000/your-image

Road trip to Persistence on CloudFoundry Chapter 2

Chapter 2

nguyen thinh

nguyen thinh

nguyen thinh

Latest posts by nguyen thinh (see all)

Road trip to Persistence on CloudFoundry

Chapter 2 – Bosh

Bosh is a deployment tool that can provision VMs on different IAAS such as AWS, OpenStack, vSphere, and even Baremetals. It monitors the VMs’ lives and keeps track of all processes that it deploys to the VMs. In this tutorial, we will focus on how to use bosh with vSphere. However, you can apply the same technique for your IAAS too. The reason we talk about Bosh in this roadtrip is because we use it to deploy Cloud Foundry.


Table of Contents


1. How it works

2. Install Bosh

  1. Create a Bosh Director Manifest
  2. Install
  3. Verify Installation

3. Configure Bosh

  1. Write your Cloud Config
  2. Upload cloud config to bosh director

4. Use Bosh

  1. Create a deployment Manifest
  2. Upload stemcell and redis releases
  3. Set deployment manifest and deploy it
  4. Interact with your new deployed redis server

1. How it works

Let’s say you want to bring up a VM that contains Redis on vSphere, you provide Bosh a vSphere stemcell (Operating System image) and a Redis Bosh Release (Installation scripts). Bosh then does the job for you.

alt text

Are you excited yet? Let’s get started on how to install Bosh on your IAAS and use it.

2. Install Bosh

In order to use Bosh in vSphere, you will need to deploy a Bosh Director VM

1. Create a Bosh Director Manifest

This manifest describes the director VM’s specs. Copy the example manifest from this docs and place it on your machine. Modify the networks section to match that of your vSphere environment.

2. Install

After configuring your deployment manifest, install bosh-init tool from this docs. Then type the following command to deploy a Bosh director VM

bosh-init deploy PATH_TO_YOUR_DIRECTOR_MANIFEST

3. Verify Installation

After completing the installation, download Bosh client to interact with Bosh director. If you have any installation problem, please refer to: docs

gem install bosh_cli --no-ri --no-rdoc

Then type

bosh target BOSH_DIRECTOR_URI

If the command succeed, you now have a functioning Bosh director.

3. Configure Bosh

To configure Bosh Director, you pass it a configuration file called Bosh Cloud Config. It allows you to define resources, networks, vms type specifically for all of your Bosh vms deployments.

alt text

1. Write your Cloud Config

To write your cloud config, copy the vSphere Cloud Config example here: tutorial and modify it accordingly.

2. Upload cloud config to bosh director

bosh update cloud-config PATH_TO_YOUR_CLOUD_CONFIG

After defining a cloud config, you are then able to deploy VMs on Bosh.

4. Use Bosh

Let’s deploy a simple redis server vm on vSphere using Bosh.

1. Create a deployment Manifest

---
name: redis-deployment
director_uuid: cd0eb8bc-831e-447d-99c1-9658c76e7721
stemcells:
- alias: trusty
  os: ubuntu-trusty
  version: latest
releases:
- name: redis
  version: latest
jobs:
- name: redis-job
  instances: 1
  templates:
  - {name: redis, release: redis}
  vm_type: medium
  stemcell: trusty
  azs: [z1]
  networks:
  - name: private
  properties:
    redis:
      password: REDIS_PASSWORD
      port: REDIS_PORT
update:
  canaries: 1
  max_in_flight: 3
  canary_watch_time: 30000-600000
  update_watch_time: 5000-600000

In this deployment manifest, I am deploying a redis VM using redis release and ubuntu stemcell. I wanted the VM type to be medium and the VM to be located at availability zone z1.

2. Upload stemcell and redis releases

bosh upload stemcell https://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent
bosh upload release https://bosh.io/d/github.com/cloudfoundry-community/redis-boshrelease

Alternatively, you can download them and run bosh upload locally.

3. Set deployment manifest and deploy it

bosh deployment PATH_TO_YOUR_DEPLOYMENT_MANIFEST
bosh deploy

4. Interact with your new deployed redis server

Find your vm IP using

bosh vms

Connect to your redis server

redis-cli -h YOUR_REDIS_VM_ADDRESS -a REDIS_PASSWORD -p REDIS_PORT

Test it

SET foo bar
GET foo

Follow Us on Twitter

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.