Author Archive

Gary White

Gary White

Gary White

Latest posts by Gary White (see all)

Kubernetes and UDP Routing

Hey Guys, Gary Here.

With all of the fun stuff happening around Kubernetes and Cloud Foundry, we decided to do some fun stuff to play around with it! One of the (few) capabilities we don’t have with Cloud foundry that we can get with Kubernetes is UDP routing.

To learn more about why UDP routing doesn’t work with the containers in Diego runtime (yet, but will), check out ONSI’s proposal for the feature.

UDP Routing. Why would you use it? In short, for applications that continually post data that isn’t important enough, or would soon be replaced with a more recent copy anyways, UDP packets can be a less intensive alternative than using the TCP routing solution. Or, if you’re really hardcore, you could implement your own verification with UDP, but that would be a blog post in itself 🙂

Overall, setting up Kubernetes and getting it to expose ports was very simple. If you are reading this without any Kubernetes setup, go check out minikube. Even better, you could set up a GCP cluster, vSphere, or (gasp) AWS and follow along. The kubectl commands should be about the same either way.

Once you’ve got your instance set up, check out our kube-udp-tennis repo on Github. We use this  repo to store very simple python scripts that accept environment variables for ports and will either send or receive messages based on which script we execute. We also baked these into a Dockerfile to allow Kubernetes to reference an image on docker hub.

Before you worry about deploying your own docker images, know that you are not required to for this example. If you were to deploy the listener, add the service link, then go ahead and deploy the server, this solution would be a working UDP connection! This is because it’s referencing our existing images already on the Docker Hub. Before I go and give you the commands, I want to explain what they do.

from /udp_listen:

kubectl apply -f udplisten-deployment.yaml

this command will go into the udplisten-deployment.yaml file, which gives the specification for our udp-listen application. We spec this out so we can extend it for the udp-listen service.

kubectl apply -f udplisten-service.yaml

this command will go into the udplisten-service.yaml file, which after the udplisten deployment has been made live, will allow us to talk into the port through the service functionality in Kubernetes. Here’s the documentation for services.

At this point, we will have the kubernetes udplisten service running, and we will be ready to deploy our dummy application to talk into it.

from /udp_server:

kubectl apply -f udpserver-deployment.yaml

This will deploy the udpserver application, and should ping messages into the udplisten-service, which you should see through the logs in the service’s pod.

The way that the udp-server.py application can find and ping into the udplisten-service is by leveraging the Kubernetes Service Functionality. Basically, when we start Kubernetes services, we will be able to find those services using environment variables. From the documentation:

For example, the Service "redis-master" which exposes TCP port 6379 and has been allocated cluster IP address 10.0.0.11 produces the following environment variables:

REDIS_MASTER_SERVICE_HOST=10.0.0.11
REDIS_MASTER_SERVICE_PORT=6379
REDIS_MASTER_PORT=tcp://10.0.0.11:6379
REDIS_MASTER_PORT_6379_TCP=tcp://10.0.0.11:6379
REDIS_MASTER_PORT_6379_TCP_PROTO=tcp
REDIS_MASTER_PORT_6379_TCP_PORT=6379
REDIS_MASTER_PORT_6379_TCP_ADDR=10.0.0.11

We search, therefore, for the udplistener_service_host and udplistener_service_port to communicate with the udplistener pods directly. Since we defined the UDP protocol as network traffic into the service, this works right out of the box!

Thanks for reading everyone, as always, reach out to us on twitter @DellEMCDojo, or me specifically @garypwhitejr, or post on the blog to get some feedback. Let us know what you think!

Until next time,

Cloud Foundry, Open Source, The way. #DellEMCDojo

Dell EMC Dojo at Hopkinton! Reviewing what we covered, and what we learned

Reviewing what we covered, and what we learned

Hey again everyone! We’re writing out today to talk about a few topics that we covered during our time at the Dell EMC Dojo Days in Hopkinton. We met with a lot of great minds and took plenty of input into how we work. We also gave a lot of insight to passersby in what we did and how we worked.

Firstly, Brain Roche and Megan Murawski debuted the famous “Transformation Talk”. We normally give this presentation in preparation for an engagement with another team, but in this case, it was given to allow open criticism and circulation of our methodology to the rest of the company. We cover in this presentation: pairing, and why it’s important to our process, why we use TDD (Test Driven Development) (and why you should too!), and our weekly meetings including Retros, IPM, and Feedback to name a few. We had plenty of great ideas and questions, as usual, and we realized twenty minutes over time that we couldn’t get Brian off the stage.

Xuebin He eventually got Brian off-stage for a talk he conducted on CICD (Continuous Integration, and Continuous Deployment). Xuebin being one of the developers at the Dojo allowed him to be a bit more technical in his talk, and cover some of the programming practices and tools we use to achieve this at the Dojo. Concourse is our tool for running our beautifully constructed tests, along with standard mocking design patterns and the code quality produced with TDD.

We picked up again on Tuesday at 1145 to talk about why a PaaS exists, and why it’s important. That talk, given by yours truly, was focused on some of the common technical roadblocks that keep developers, customers, and managers from being able to work efficiently; as well as the ways using a PaaS can solve those problems to build a better business.

To containerize applications for a PaaS, we would need to learn basics like “What is a 12 factor application, and what’s a container?” Thinh Nguyen stepped in and gave a great description on how we use guiding principles while developing our application environment to be better for us and our customers.

Throughout all of our talks, we worked away on two pair stations very carefully brought from our lair in Cambridge. We gave away some free swag, some free candy, and raffled off some super giveaways. We thank everyone involved in preparing and executing these few days for their hard work. We also want to give a huge thanks to everyone who attended our talks (rambles) and participated in some mind-expanding conversations.

Finally, I want to close with a few notes. We always enjoy fresh perspective. If you had more to say, or you missed us during our time and you want to start a conversation, leave a comment in the comment section! If you don’t want to comment here, then drop us a line in my email. We’d love to hear from you.

Until next time, remember: Cloud Foundry, Open Source, The Way. #DellEMCDojo.

A Fresh Developer’s take on Cloud Foundry Why the third platform is the new way to iterate

Why the third platform is the new way to iterate

Coming to the Dojo a little over two months ago for my first day of work at my first (out of college) job, I was mystified by the amount of understanding and knowledge I still had to gain by using the Cloud Platform as a development and deployment environment. After the first few weeks, we started developing a brand new application for internal use. There was no existing infrastructure, but a clear idea to use a SQL database, and a web portal for the UI. I was thrilled! I worked on my fair share of web applications, so this would be a cake walk.

Great! What were the IP’s? What was the endpoint? What was the downtime for our environment on updates? What was our server technology? They said it was up to me. I began thinking of asking IT for a VM spin up our database, scripts designed to back up the data with cron jobs, registering a DNS for our website, and for a potential duplicate set for our test environment; technical aches and pains that are first steps when building a web application. I then began asking what seemed the obvious question of who to go to for the resources, and I was told we would do it all ourselves. Wait, I thought, we were supposed to deploy and maintain the VM’s set up users for the database, download the applications, and backup technology? That was easily another two or three days short term, and days of complications long term. Or so I thought.

Except…I hadn’t known that we were using Cloud Foundry, and the benefits that came from this. All of the pain points I used to face in setting up a functioning web app were moot. Instead of spending time configuring and maintaining environments, the Dojo, as a team, has put together a full-functioning web application with production environment, architecture, an extensive automated testing harness, and automated deployment. All as painlessly as we might use Github.

Removing Development Roadblocks

When I thought of the Web application, a typical structure comes to mind of the architecture for the app.

Screen Shot 2016-08-31 at 12.20.49 PM

A typical developer headache for a web application

There’s plenty of steps for our team, as developers, to maintain and care about here. But with Cloud Foundry, we can remove many of these concerns.

Lets talk about Databases first. Using service brokers as provided through CF, we can simply bind a SQL service to our application. For extra sweetness, we can use a buildpack that supports that our server is connecting to a service-provided database. The buildpack will, at runtime, connect our application’s backend into a bound SQL database, no environment variables necessary.

Servers and UI? No problem. Pushing applications to Cloud Foundry takes as many commands as pushing to git. We can also update and configure the application directly through a UI included in our environment. This means changing something we forgot to put in our three-four deployment commands doesn’t warrant another deployment. As far as firewalls go, using the proper security groups to avoid exposing secretive endpoints is a few minutes behind the scenes of a UI, or a couple commands in a terminal.

Alright, so our development roadblocks are covered. What does all this mean for our Customers?

Rapid Integration and Feedback Loops

With our Test Driven Development and Continuous Integration mantras, we can rapidly iterate on our products. With Cloud Foundry, those rapid iterations allow us to take customer feedback. Using our real product on something as quick to set up and easily accessible as CF platform is, we can respond to our customer’s concerns much more efficiently.

Allowing our customers a way to directly interact with our product in this way is not only helpful for them, but for us, for a variety of reasons. Firstly, we’re able to get valuable feedback from them in a timely fashion, not when a feature has been done and untouched for two months (less old code touching, the better). Additionally, we can test what our real runtime environment will be like for our production level product, which means faster satisfaction for customers.

Screen Shot 2016-08-31 at 12.48.14 PM

Our development cycle. Made simple with less setup behind the scenes

Finally, we don’t have to worry about maintaining servers and VMS when they may go down. Cloud Foundry can sense when an app crashes, and restart it for us. Same database, same DNS, and limited downtime. Doesn’t get much easier than that.

Final thoughts

Using Cloud Foundry also means we can simplify our development process by having a consistent API and well-documented environment surrounding our application. When we rotate around our projects, people coming into our web application are using similar knowledge and practices they might use when deploying our Minecraft Docker Containers.

It’s for the reasons in this article, and I’m sure for many more to come, that the cloud platform and Cloud Foundry has made development easier for us. It allows us to create better products, faster. We can get immediate feedback from customers and get stories accepted faster. For most developers, that’s a no brainer.

Follow Us on Twitter

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.