Author Archive

Sneak Preview: Cloud Foundry Persistence Under the Hood How Does Persistence Orchestration Work in Cloud Foundry?

How Does Persistence Orchestration Work in Cloud Foundry?

A few weeks ago at the Santa Clara Cloud Foundry Summit we announced that Cloud Foundry will be supporting persistence. From a developer’s perspective, the user experience is smooth and very similar to using other existing services. Here is an article that describes the CF persistence user experience.

For those who are wondering how Cloud Foundry orchestrates persistence service, this article will provide a high level overview of the architecture and user experience.

How Does it Work when Developer Creates Persistence Service?

Like other Cloud Foundry services, before an application can gain access to the persistence service, the user needs to first sign up for a service plan from the Cloud Foundry marketplace.

Initially, our Service Broker uses an Open Source technology called RexRay, which is a persistence orchestration engine that works with Docker.

Screen Shot 2016-06-07 at 11.02.31 AM

Creating Service

When the service broker receives a request to create a new service plan, it would go into its backing persistence service provider, such as ScaleIO or Isilon, to create a new volume.

For example, the user can use:

cf create-service scaleio small my-scaleio1

Deleting Service

When a user is done with the volume and no longer needs the service plan, the user can make a delete-service call to remove the volume. When the service broker receives the request to delete the service plan, it would go into its backing persistence service provider to remove the volume and free up storage space.

For example, the user can use:

cf delete-service my-scaleio1

Binding Service

After a service plan is created, the user can then bind the service to one or multiple Cloud Foundry applications. When the service broker receives the request to bind an application, the service broker would would include a special flag in the JSON response to the Cloud Controller, so that Cloud Controller and Diego would know how to mount the directory in Runtime. The runtime behavior will be described in more details below.

 

How Does it work in Runtime?

Cloud Foundry executes each instance of application in container-based runtime environment called Diego. For Persistence Orchestration, a new project called Volman (short for Volume Manager) has become the newest addition to the Diego release. Volman is part of Diego and will live in a Diego Cell. At a high level, Volman is responsible for picking up special flags from Cloud Controller, invoke a Volume Driver to mount a volume into a Diego Cell, then provide access to the directory from the runtime container.

Screen Shot 2016-06-07 at 12.21.18 PM

 

How to Deploy Diego in a Hybrid Environment Deploying Cloud Foundry Diego in a Virtual and Bare Metal Environment

Deploying Cloud Foundry Diego in a Virtual and Bare Metal Environment

In my last post “Deploying Cloud Foundry in a Hybrid Environment“, I showed you how to deploy DEA runners in a bare-metal environment. However, Cloud Foundry now has new and exciting Diego runtime! So now I will guide you through how to deploy Diego Cells on bare metal machines. When we’re all done, your environment will look like this:

Diagram7

(more…)

Creating Cloud Foundry Applications with Persistence Traditional & Cloud Native Applications with Persistence

Traditional & Cloud Native Applications with Persistence

Cloud Foundry and 12-Factor applications are great to create Cloud Native Applications and has become the standard. But you and I don’t just have to worry about our new 12 Factor apps, we also have legacy applications in our family.  Our position is that your traditional apps and your new cool 12-Factor apps should be able to experience the benefits of running on Cloud Foundry.  So, we have worked to create a world where your legacy apps and new apps can live together.

cf-logo

In the past, Cloud Foundry applications cannot use any filesystem or block storage. That totally makes sense, given that Cloud Foundry apps are executed in elastic containers, which can go away at any time. And if that happens, any data written to the local filesystem of the containers will be wiped.

If we can externalize persistent storage to be a service – and bind and unbind those services to Cloud Foundry applications – a lot more apps can run in Cloud Foundry. For example, heavy data access apps, like databases and video editing software, can now access extremely fast storage AND at the same time experience the scalability and reliability provided by Cloud Foundry.

Allowing Cloud Foundry applications to have direct persistence access opens a lot of doors for developers. Traditional applications that require persistence can migrate to Cloud Foundry a lot easier. Big Data Analytics applications can now use persistence to perform indexing and calculation.

Traditionally, a lot of data services consumed by Cloud Foundry applications, such as MySQL, Cassandra, etc, need to be deployed by Bosh as Virtual Machines. With Persistence, we can start looking bringing these services to run in Cloud Foundry or create the next generation of Cloud Native data services.

What can Developers Expect?

When developers come to the Cloud Foundry marketplace by using cf marketplace, they will see services that can offer their applications persistence:

Screen Shot 2016-05-20 at 10.37.37 AM

The details of the service plan can be seen by cf marketplace -s ScaleIO

Screen Shot 2016-05-20 at 10.57.04 AM

This is a plan that would offer 2GB of storage for your Cloud Foundry applications. Let’s sign up for it by running cf create-service scaleio small my-scaleio-service1

Screen Shot 2016-05-20 at 11.02.26 AM

By creating a service instance, the ScaleIO Service Broker goes into ScaleIO and creates a volume of 2GB. We are now ready to bind this new service to our app. To demonstrate the functionality, we have created a very simple application that will write and read to the filesystem:

Screen Shot 2016-05-20 at 11.09.51 AM

After a cf bind-service call, the storage will be mounted as a directory and the path will indicate an environment variable inside the service. For example:

Screen Shot 2016-05-20 at 11.35.07 AM

Based on the container_path variable, the application can read and write as if it’s a local filesystem.

 

How to Deploy Cloud Foundry in a Virtual & Bare Metal Hybrid Environment Deploying Cloud Foundry on VSphere and Bare Metal

Deploying Cloud Foundry on VSphere and Bare Metal

This is a guide for setting up Cloud Foundry in a hybrid environment with virtualization and bare-metal infrastructure. Cloud Foundry is made up of many components. It makes sense to run most of them in a virtual environment for resource sharing and separation reasons. And these pieces might not require the resources for an entire physical machine all to itself.

Computing components, such as DEA Runner or Diego Cells, should run on Bare-Metal to fully make use of the computing power, without spending resources for your virtualization tier.

This tutorial will guide you step by step, to set up Cloud Foundry components on vSphere and DEA Runner on Bare-Metal machines. When you finish, your environment will look like this:

Diagram1

(more…)

Follow Us on Twitter

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.