Head in the Clouds

Interested in building elastic, scalable and profitable open source clouds? Join Citrix for free day of learning, best practices and industry insights at the Build a Cloud Workshop in San Francisco, CA on May 10, 2012.  This free one-day workshop is held alongside with Citrix Synergy 2012 and includes a pass to the Synergy Keynote and Solutions Expo. This event offers a unique opportunity to learn about the cloud building as well as the challenges and successes of delivering cloud services.

• Learn how to build an open source cloud with solutions from Citrix CloudStack, Canonical, RightScale, Zenoss and Xen.org.
• Network with peers, industry experts and solution providers at the CloudStack Community Reception

The Build a Cloud Workshop is held at the Westin Market Street (Metropolitan Ballroom 1 and 2, 2nd Floor) located at 50 3rd St, San Francisco, CA 94103. Click here to learn more and register.

Build a Cloud Workshop Agenda

8:30 – 1:00 Optional            Free admission to the Synergy Keynote,Synergy Solutions Expo and lunch

1:00 – 2:00                           Introduction to Cloud Computing Mark Hinkle, Sr. Director Cloud Community

2:00 – 3:00                           Virtualization in the Cloud Lars Kurth, Xen.org Community Manager

Since we made the announcement of licensing and governance shift a few weeks back we've been inundated with requests for developer support from individuals and companies who want to develop, extend, integrate with or just hack on CloudStack. Lots of those requests were coming with requests to visit developers in person to get up to speed as rapidly as possible. After setting up several of those meetings in person, we realized it wasn't beginning to sate demand, so in reaction we decided to run a two day event decided to rapidly on-ramp folks, we put the agenda in the hands of two of CloudStack's lead developers/architects. What came out of that is the 'Developer On-ramp - two days of no-fluff CloudStack development and integration - lots of hands on work.. It's free to attend, you just need to register here.

Agenda:

Day 1: Everyone

9:30-12:00 - General CloudStack Architecture
12:00-13:00 - Lunch
13:00-14:00 - Storage deep dive
14:00-15:00 - Failure modes, H/A, maintenance planning
15:00-15:30 - Break 
15:30-17:00 - Networking deep dive
17:00-17:30 - Show and tell - see what others in the community are working on
17:30-18:30 - Beer

Day 2- Integrators

9:30-12:00 - CloudStack API deep dive.
12:00-13:00 - Lunch
13:00-14:00 - Development environment access 
14:00-17:00 - Building a sample app (bring your own ideas if you want)

Day 2 - Core developers

9:30-10:30 - Setup to contribute (git, IDE, patches, devel environment)
10:30-12:00 - Writing tests, simulator
12:00-13:00 - Lunch
13:00-14:00 - Hypervisor and bare metal management
14:00-15:00 - Scalability
15:00-15:30 - Break
15:30-17:00 - Freeform hands on hacking time - (bring your own ideas or squash bugs)

Recently, Citrix announced that it will donate CloudStack to the Apache Software Foundation (ASF).  This is a very exciting moment in the history of the CloudStack project.  The ASF is the premier open source foundation in the world and it has a track record of successful, completely open projects.  The CloudStack project has been increasingly open during its life.  But, “increasingly open” isn’t good enough.  We have to be 100% open in our technical discussions to run the project in the meritocratic fashion that successful Apache projects use.  In this blog post I’ll do a brief review of CloudStack openness and list out the changes we’ll be making over the next few months to transform CloudStack from a largely single-entity development effort to a collaborative, community project.

CloudStack 1.0 was proprietary software.  Closed specs, closed discussion, closed bug database, closed source.  The CloudStack 2.0 release in May, 2010 transitioned CloudStack to an open-core model.  About 95% of the source was available under GPLv3 and the bug database was open, but the technical discussions were closed and few technical documents were publicly available.  In August, 2011 Citrix published 100% of the CloudStack source code.  We started publishing some technical specs, but the discussions were still closed.

In late 2011, in conjunction with the Acton release, we started doing a better job publishing technical specs.   Not perfect – we weren’t iterating the designs in the open – but good progress.  We also (finally) came up with a process for accepting contributions.  In 2012 we’ve created a publicly available project dashboard , started publishing meeting minutes, and getting a larger number of non-Citrix contributions.

That’s good stuff, but there are some fundamental changes needed to make it more open.  Instead of publishing designs and discussion results after the fact, we have to publish the earliest version and then iterate in the open.  We also have to have the technical discussions on the cloudstack-dev mailing list (see below for the new one), and not on our internal engineering list.  This will take policing -- and I’m part of the problem here -- but through force of will a few folks can make a big difference here.

We also need to ask more of the community.  Our bug database is open, but, there’s no place to go with an “ask-list”.  Nowhere can you easily see important features with no owner, nor can you easily search to find the bugs with the most votes.  We need to pick a few features with a variety of technical needs and ask the community to help out.  That part is easy, BTW.  CloudStack has so many potential integrations in the datacenter that just listing those would give years of work to interested parties!

Today Citrix announced that CloudStack would become the cloud platform project in Apache Software Foundation. I’m excited not just because CloudStack will be an incredibly vibrant and successful project by itself, I also believe there is a tremendous amount of synergy between CloudStack and other cloud-related projects in Apache Software Foundation. I look forward to continuing to work with, for example, Apache Libcloud and Deltacloud projects.

I am the most excited, however, about the prospect of integrating with Apache Hadoop project. Known primarily as the technology for Big Data applications, Hadoop has gained wide-spread adoption in the industry. Similar to CloudStack which is inspired by Amazon’s EC2 service, Hadoop is modeled after Google’s MapReduce and Google File System technologies. And just like CloudStack, Hadoop is implemented in Java.

At the lowest level, Hadoop Distributed File System (HDFS) is a distributed and scalable file system. HDFS is designed to run on a large number of hosts and achieves reliability by automatically replicating data across multiple hosts. Hadoop project also includes a MapReduce engine and HBase distributed database (modeled after Google’s BigTable.) MapReduce and HBase run on top of HDFS. Highly reliable and highly efficient, Hadoop technology is being used by some of the largest cloud companies including eBay, Yahoo! and Facebook.

Today, CloudStack users already run Hadoop on CloudStack. They implement a service very similar to Amazon’s Elastic MapReduce (EMR). For cloud service providers, Hadoop represents a significant amount of workload that can be readily moved to the cloud. Enterprise deployments can achieve tremendous savings by leveraging the same CloudStack infrastructure to host Big Data workload. Users also leverage CloudStack’s bare metal provisioning capabilities to build high performance Hadoop clusters.

Working closely with Hadoop development community, we have started to explore other ways to integrate CloudStack and Hadoop. Because of its scalability, reliability, performance, and maturity, HDFS is a great object store solution for IaaS cloud. We have started the development of an S3 API front-end for HDFS. Once that work is complete, the combination of CloudStack and Hadoop will provide features equivalent to Amazon EC2 and S3 services.

When CloudStack was launched as an open source project back in 2010 our goal was to make CloudStack the leading open source cloud computing platform and to enable our users, partners, and customers to easily use, modify, and contribute to the project. We chose to license the project under the GNU Public License v3 (GPLv3) then.  The reasoning was that this was not only an OSI-approved license but also it allowed us to maintain compatibility with the Apache Software License. At the time it was the right move for the project... but times change.

Today we are announcing our move from the GPLv3 to the Apache License 2.0. We are also planning to apply contribute the CloudStack project to the Apache Software Foundation(ASF) and with their acceptance to turn CloudStack into a truly community run project but with governance provided by the Apache Foundation rather than by Citrix alone.  In addition, Citrix has stepped up its investment in the project and the open source community and decided to become a platinum sponsor of the Apache Foundation. They also will be hiring additional developers, evangelists and providing other resources to help insure the success of the CloudStack project in the long-term.

Why Switch Licenses?

Over the course of the last two years the feedback from our users and partners was that while the GPL was a fine license that it did throw up some roadblocks to their participation and use. Today Citrix has announced their intention to contribute the CloudStack code to the Apache foundation and to license CloudStack 3.0.1 and all future versions under the  Apache Software License.

This decision was not made lightly and frankly when the idea was floated I was a little skeptical as a fan of Linux I was fond of saying, “If the GPL is good enough for Linus, it’s good enough for me.” The problem is that I am only one person and while it was fine for me it was not the license that best served everyone in our community and put up barriers for others to participate.