Oct 22, 2016

Scaling your MQTT Deployment from Trial to Millions of Clients

By 2020, industry experts are projecting over 20 billion Internet of Things (IoT) devices will be deployed.  To accommodate IoT devices with low power and bandwidth requirements, organizations are using MQTT, a standard lightweight protocol for IoT communication.

MQTT shifts today's request/response web paradigm to an event-driven publish/subscribe architecture which is a perfect fit for IoT use cases. Supporting a large number of MQTT devices requires enterprises to build infrastructure with fully automated operations that can grow from a single data center to a hybrid multi-cloud environment.

This webinar focuses on scaling MQTT infrastructure and includes use cases showing enterprise and cloud-based deployments.  It answers the following: 
  • What are the MQTT scaling challenges?
  • How is MQTT broker capacity scaled within a datacenter?
  • What are the challenges in load balancing: MQTT over TCP? MQTT over WebSocket?
  • How can changes (e.g. new broker) be dynamically managed in MQTT environments?
  • How does load balancing improve MQTT broker availability and performance?
  • How do clients seamlessly access MQTT brokers across a hybrid cloud?
  • What are examples of MQTT deployments?
Dominik Obermaier, CTO, HiveMQ
Dore Rosenblum, VP Marketing and Products, Elastic Beam

Register for the live event: https://mqttscaling.clickmeeting.com/mqtt-scaling-webinar/register
Wednesday, November 9, 2016 | 9AM PDT 

Sep 1, 2014

Linux Containers - Server Virtualization 2.0 : A Game Changer!

Summer Greetings!

The highlight of this blog is about Linux Containers and how it will once again change the server computing by taking us back to the roots of running single OS/server! First, let me quickly highlight the history of Linux Containers and how it evolved over a period of time.

Linux Container History:

  1. 1979: chroot was introduced in Unix7
  2. 1982: chroot added to BSD by Bill Joy
  3. 2000: chroot expands to jail cmd in BSD
  4. 2005: Solaris introduces zones
  5. 2008: Linux introduces lxc. (user space tool to create containers)
  6. 2013: namespaces inclusion in Linux kernel 3.8 
  7. 2014: Container popularity and toolkits like Docker built on top of lxc makes it easy to template/automate apps inside containers

1) Why Server Virtualization? (2000: Unix)

Let us first to take a look at why servers are virtualized using hypervisors like ESX(VMware) or Xen/KVM. The primary reason was "under utilization" of servers. If I remember correctly from my Sun days, it was around 30-40%! The secondary reason is entry of Windows Server OS and its memory leak problem in the servers.

The hypervisors helped to fix both problems:
a) Run many Guest OS thereby running many applications
b) If there is a problem with Windows Server OS, safely reboot it without affecting other applications

Sun did have an answer for under utilization using Solaris Zones. Unfortunately Sun lost its way and Linux had very poor implementation of containment features around 2005. So hypervisors are the only way to increase the server utilization and it resulted in huge success for VMware followed by EC2 in AWS.

2) Server Virtualization 1.0 (2004: Hypervisors)

VMware done a good job of increasing server utilization by enabling to run many OS per server to fully utilize the server hardware using hypervisor technology(ESX). This success resulted in other hypervisor products such as Xen/KVM, Hyper-V.

Here is the overview of server virtualization using hypervisor:

OK. Now let us look at the issues with this approach.

a) CPU
It's slow.

It's going through 2 OS layers before apps are getting executed in CPU. Naturally products like App Servers and Databases are not suitable to run on this type environment. To fix this, there is a technique called "Para Virtualization" which was invented to run OS system calls directly on hardware. Here is the problem: OS needs to be modified to support underlying hypervisors. This creates OS incompatibility- we were handling 3 types of RedHat 6.4! (Bare Metal, Full Virtualization & Para Virtualization)

Then there are some fundamental problems like clock issues in Redhat OS running on EC2 in Para Virtualization mode! Once EC2-S3 communication failed because of clock/NTP problem. Can you believe this type of clock issues in 2014? But that's the reality of running too many abstractions!

b)File System
It's slow. IIOPS are terrible in hypervisor environment. In this masala-curry type environment, we need to create distributed file systems to share data across multiple VM's and adds another level of complexity!

The web products are fundamentally distributed systems. They will be speaking many protocols like HTTP, JDBC, AMQP, etc over TCP to create a single application. This type of super-abstraction naturally slows down packet flows!

Finally for each VM we will be duplicating libraries! You can see in the picture above. It's fatty in the range of few GB's, so ability to move around VM is also slow. More importantly since it's fatty we can pack only less number of applications inside a server.

Sure, from the CIO perspective he got maximum server utilization. But how did he get it? Running OS on top of OS and duplicating libraries!!! It's simply low performance and high cost environment.

Note: Hypervisors are based on emulating virtual hardware

3) Server Virtualization 2.0 (2014: Linux Containers)

Remember Sun was trying to improve utilization using zones? The same type of concept is now available in Linux kernel, mainly thanks to all the parties including Google agreeing to create a single "containment features" inside kernel. (Unlike KVM/Xen) It is now officially called as Linux Containers. The linux container basically allows to create many applications in a single OS running on a server. The server utilization, isolation/security is provided with high performance with less abstraction!

Container Ecosystem:
Docker(toolkit)-> LXC(user space)-> Linux Container(namespace + 

The 2 primary Linux containment features are CGroups and Namespaces.

In simple terms, linux containers are about grouping of processes and assign network + file system. This enables to run many applications inside the same server using single OS.  It completely removes hypervisors, dual OS, duplicating app libraries and more importantly allows apps to run directly on hardware instead of system call translations.

Developers can develop, test and run in the same environment. There is no OS incompatibility issues and helps for faster software development.

Here is the overview:

You see here, it's simple! One OS and one set of libraries for all the applications. So we can pack more apps, higher performance and lower cost.

It make sense? Now the question is how real is Linux Containers. The short answer is that everything at Google runs on Linux Containers- search engine, gmail, etc. Google engineers are saying that 2 billion linux containers are created per week! The following picture illustrates the evolution of containers at Google:

Linux containers are already in production. Linux community has agreed to create one technology for containers inside kernel. Toolkits like Docker are helping to create containers. It's real.

Note: Containers are based on sharing operating system kernel


Simplicity always wins in the computer industry. Linux container will take mainstream in next few years. We will see more cloud providers switching to Linux container based model from hypervisor model in the coming years. It's all about high performance and low cost. Linux Containers as the new paradigm has just started- design your systems specifically to take advantage of containers!

There is no question that legacy hypervisor based virtual machines will exist for a while and we haven't even seen the end of life for mainframes! The growth of hypervisor based virtual machine will stall first and then decline. You can already see that VMware started supporting container and will be interesting to see how they position with ESX which is their cash cow.

High Performance Technical Computing and High Performance Business Computing will jump to container technology first as it helps big time! (Google running everything on container is a good example)

Jun 29, 2014

SaaS on Cloud vs SaaS on Datacenter

Summer 2014 Greetings!

This blog is based on my direct experience of building both SaaS on AWS and  SaaS on Datacenter.

Datacenter: Rent a floor space from someone like CoreSite
Cloud: Rent IaaS from someone like AWS

It is important to understand the key difference between these 2 approaches. There are pros and cons for each approaches. My suggestion is to build SaaS on Cloud first until there is a product/market fit. So I will be biased for SaaS on Cloud in this discussion!

SaaS on Datacenter

It is a natural thinking to control the full fabric but reality is hard, if we have to build a SaaS on Datacenter from get-go. This is an old-school architecture and hard labor.
  1. How much Investment? 
    • We are looking at few million $'s, at least a million to start with... 
    • On top of $, time and energy needed to get a datacenter operational is not an easy task especially for new SaaS startups.
    • Instead of focusing on building business logic, we will be worrying about summer heat impact on server cooling OR harsh winter snow storm on complete power outages!
  2. How many Datacenters?
    • It's a global world. We can't build datacenters around the world based on customer demand!
    • Even more important is that we may not even know how many we need until we start acquiring customers
    • If we keep US-only OR Europe-only, network latency will kill the applications for rest of the world
  3. Where to Start?
    • We will be quickly overwhelmed with dealing with many vendors(server/storage/network), ISPs, DNS issues, hiring system administrators, managing local data/privacy laws, etc
    • Operations engineering will front end software engineering and product focus will get lost!
Trust me, operations engineering is harder than software engineering! SaaS is a distributed system and if the underlying infrastructure is not stable, the entire system will fall apart quickly! Speak to guys who have dealt with production cloud services (Amazon.com, LinkedIn, Salesforce, etc) and they will tell you the stories/nightmares!

Having said all of them, you can think about this approach when your SaaS revenue reaches $30M/year. Then it will make sense to control full fabric and lower cost.

SaaS on Cloud

There is no question that we will depended on cloud providers for stability, performance and availability. It is important to establish right relationships before committing to build product on top of it. "Cloud-neutral" is hard but should be the mantra for your SaaS architecture.
  1. Where to Start?
    • AWS (simple)
    • There are other choices: Azure, Google Cloud, IBM Softlayer, Rackspace, Century Link, etc. Just take a look at this Gartner cloud magic quadrant for May 2014.
    • Talk to folks in your network who have done it! Don't assume anything and you will be surprised about how hard it is to run a cloud service 24/7. (technical advisory board is recommended)
  2. How many Cloud-Datacenters?
    • 3 for sure (North America, Europe and Asia)
    • You can add more if data privacy becomes an issue (like China or Europe)
    • Multiple cloud vendors is recommended only for 1.0 product release
  3. How much Investment?
    • $5K/month/POD (POD: single instance of complete application with a unique URL)
    • This is for high end SaaS in AWS. You can reduce it. Do not have to worry about hiring big operations team, data backup/recovery, etc.
    • Scale cloud-cost only based on business demand. (auto scale is your friend)
Buy a big world map and put it on the wall. Put the engineering person on the left and product person on the right. Start circling cities on the map to identify potential users of your SaaS. You will quickly realize that it's a small market, no matter what you told your investors!

Then you will realize that your potential customers are spread around the world!  So "SaaS on Cloud" is not only handy but also will make you smart in your board room discussions :-)


The clouds are like beaches! If you go to the beach on Friday afternoon, it will be empty and clean. But on Saturday morning it will be dirty and crowded. The cloud vendors are multi-tenant and are sharing resources. You will face this problem and will notice it when there is an event like the FIFA World Cup or Thanksgiving holidays. Then there is mother nature like this one. Plan ahead. Keep an eye on world events and mother nature.

I had a good experience with AWS California region in terms of stability. It is not crowded and not much problem with heat or cold. But if there is a big earthquake then it is a different story!

The summary is that start with SaaS on Cloud and move to SaaS on Datacenter if needed. Have fun in building next Workday or Ciespace or Netsuite! Reach out to me if you need more wisdom.

Note: IaaS vs PaaS is for another blog.

Jul 29, 2013

Google SPDY vs WebSocket Protocol

Summer Greetings!

It's been while since I wrote my last blog. Obviously doing a startup is pretty intensive and is more if it's a HPC cloud platform! (commercial: check out www.ciespace.com for our product)

The HTTP protocol is the core of the web today and it touches millions of people all over the world, no matter what country/language on a daily basis. But it has got it's own problem as the original design was to transport simple text over the wide area network. Now it's over-used or abused and clearly needs a fix. The problem is the huge success of HTTP itself- you can't simply rip and replace something like HTTP. So how do you fix it? The simple answer is to complement with additions to fix the broken pieces.

The Google SPDY and WebSocket protocols are complimentary to each other and to HTTP. SPDY and WebSocket are are fixing 2 different problems in the HTTP protocol. There are some elements that may overlap each other but the core design is different.

Le's take a look at all 3 protocols - HTTP, SPDY and WebSocket.

1) HTTP Protocol
Typical web page needs 10's(even 100's) of resources(text, data, images, audio, video,etc) to construct and display to the end user. The client has to make separate HTTP request to server to get each resource. If the web page needs 25 resources then client has to make 25 separate HTTP requests over TCP connections. This is inefficient. There are some techniques like HTTP-pipelining available to multiplex requests but it's of limited use. This network latency creates a poor user experience.

2) Google SPDY Protocol
To fix the above problem of sending multiple requests to get multiple resources from the server, Google folks introduced a session level protocol called SPDY to group many requests in a single connection! So instead of making 10 different separate HTTP requests to server, SPDY can help to put all 10 HTTP requests in a single TCP connection. SPDY helps to reduce network latency by grouping HTTP requests.

Here is the diagram from Google:

In a nutshell, SPDY is a session level protocol. It's a layer below HTTP and above TCP. SPDY "groups HTTP" requests and sends over a single TCP connection. So it's clear that SPDY is a session level protocol for HTTP and enables grouping of HTTP requests.

3)WebSocket Protocol
HTTP is a request/response protocol. If you want to stream data (text & binary) bidirectionally over a single TCP connection with much less overhead, then the answer is WebSocket protocol. The WebSocket protocol starts with HTTP connection between client(browser) and server. Then the connection gets upgraded to WebSocket (technically it's getting downgraded to TCP level for browser/server communication) for bi-directional streaming of data. So it's clear that WebSocket is an extension to HTTP and enables streaming of data.

Here is the network diagram:

HTTP - Request/Response Protocol
SPDY - Session Level Protocol( to "group HTTP" requests to reduce network latency)
WebSocket - Extension to HTTP protocol for bi-directional communication between client and server

Many Google properties are already using SPDY and end users may not even know about it! Other leading web properties like Twitter, Facebook, etc are starting to support SPDY. Most modern web browsers (Chrome, Firefox, Safari, IE) are all supporting SPDY now.

It's the same with WebSocket. Many web properties including some wall street apps are usingWebSocket protocol. All the modern web browsers supports WebSocket now.

There is no need to explain about the mobile ecosystem adopting these 2 great protocols. They are on fire now!

Welcome SPDY and WebSocket! Good bye, HTTP!

Nov 25, 2012

Why SaaS?


The temperature has cooled down here in North America and winter is around the corner. It is still relatively warm and in the cold weather it's nice to write about some warmness, which is SaaS! Sure, everyone knows the latest success of Workday. The question is why or how SaaS is succeeding in the marketplace? This may not be true for all applications but definitely business applications are going towards SaaS model.

IMO, there are 3 main factors for the success of SaaS:
  1. Runtime Environment
  2. Data
  3. Mobile
Let me first highlight few points from a SaaS innovators summit that I recently attended here in San Francisco. This summit included select group of technology executives, VC's and CIO's. The tech executives are all pro-SaaS and no need to highlight anything. One investor made an interesting comment: from Sausalito to San Jose, I make investments only in SaaS for enterprise applications!

I will highlight the CIO panel here and if you want to hear more about the summit, please ping me separately. (uday dot s at-sign comcast dot net)

There were 4 CIO's from big corporations. The moderator started the discussion asking about what % of applications are SaaS today? The answer was 90%, 50%, 25% and 20%. Then moderator said it makes sense to ask first 20% and 25% CIOs about SaaS adoption. For everyone's surprise they both said they are going to adopt SaaS aggressively in next few years!

  • 20% CIO
    • I am sick of having repeated discussion with CFO about "do more with less". If it is SaaS, its just a binary discussion. I just need to pick the right vendor. I am also sick of dealing with servers to DBA!
  • 25% CIO
    • I don't expect any IT application developers in my organization in 5 years from now. They will be doing something else! 
  • 50% CIO
    • He was interesting. His acceleration from here onwards going to be cautious. He is taking safer approach/too much legacy and will have hybrid environment. But he was super supportive of SaaS too.
  • 90% CIO
    • He said, don't ask me when is the next 10%. I am just wearing CIO hat for this panel as my official title is already changed! I don't have any legacy and so it was easy for me to take SaaS route from the beginning.
The common theme across all 4 CIO was:
  1. Support of SaaS model.
  2. Need to handle legacy environment and migration is taking time.
  3. SaaS integration is painful even today. We need a solution that can integrate many SaaS apps.
  4. Not outsourcing Active Directory any time soon.
  5. BYOD is real.
This event was in 48th floor in TransAmerica Pyramid in San Francisco (lovely conference room- if you are on the window seat and you can turn towards wonderful bay-view if the speaker is boring!). After the event I need to take 2 elevators to come to ground and another 12minutes cold/crispy late evening walk to garage to take my car. All along I was thinking about one thing: "IT ecosystem is changing for ever".

OK. Lets get back to the 3 factors that I highlighted in the beginning of this article.
  1. Runtime Environment
    • Before, the product team builds a software product on a particular OS. Then they have to port to different OS to support wider market. This is pain. 
    • Now, the software products are distributed applications. It no longer runs on a single desktop or a single server. You have a complex network environment with distributed servers and storage systems as runtime environment. It is not an easy task to reproduce the application in another environment for deployment. This is more painful. Here is an example of politely saying 'no': http://bit.ly/U3RN9C 
    • The summary is "single runtime environment" is a blessing for product teams in SaaS. It is even more blessing for IT team as they don't have to deal with software upgrades as there is no software in the first place!
  2. Data
    • In the past, all software product teams have to deal with sample applications and seed-data to perfect their software. Then they will discover "real bugs" in the customer environment! Then release service pack after service pack to fix bugs. It was a painful experience from product teams to marketing to sales to customers.
    • In the SaaS model, product team gets "real data" and "real use cases". Then they discover bugs in their own backyard (data center) and quickly fix it. The pain stays with the product team itself and nature forces them to perfect the application quickly!
  3. Mobile
    • The enterprise is increasingly accepting BYOD for accessing any-service, any-time, any-where in any-device model. This model aligns well with SaaS than behind the IT firewall with VPN challenges.
Ok. Good. Let's take a step back and ask the 2 Marc Benioff's questions:
  • In 2000: Why enterprise software is not like Amazon.com?
  • In 2009: Why enterprise software is not like Facebook.com?
My answer is that software teams were frustrated with many runtime environments and lack of data to perfect the product quickly in the old enterprise software model.

If you are still not sure about SaaS, please read this article:
"What went wrong? U.S. Airforce blows $1 billion on failed enterprise software"

The SaaS is clearly succeeding in the business applications. The enterprise IT is going through a major transformation now. The IT has reached an inflection point now with SaaS. The role of CIO is going to be even more critical for any business to succeed in this globally competitive environment. CIOs are  dealing with BYOD to legacy to SaaS to data security and more! There will be many, many applications that will continue to run inside the firewall. There is no question about it.

May 20, 2012

Apache Cassandra Database


Note: I like innovative products that solve customer problems. Cassandra is an innovative product. So I like it. The summary of this blog is that "Cassandra is yet another database" with different design principles.

Caution: If you are having data-scaling problems in your application, blindly moving to Cassandra is NOT going to solve it for you!. It's a 1.0 product and you should evaluate it carefully with your needs before you commit to it.


The title of this blog may be confusing to you, if you have "read-too-much" about NoSQL. Let me tell you- Cassandra is a database. Period. There is nothing wrong in calling it as database. I would even say that calling Cassandra as NoSQL is even insulting to it's capabilities. It has richer + sophisticated data structures for applications.

What is database? 
Database is a piece of software that organizes data and makes it available for applications.

There are many types of database and RDBMS is *just* one of them. Unfortunately/fortunately, RDBMS was so successful in the past for a very long time(3 decades), the name database was hijacked to refer only to RDBMS.

So, what is Cassandra?
Oracle  is a "row-oriented" database.
Cassandra is a "column-oriented" database.

All I am trying to say here is that Cassandra is a database and there is nothing wrong calling it a database!


To build any serious application, you need query language. You can call them what ever you like but you need it. You can't build app against *just* key-value store. Cassandra is not just key-value store. In the same context, Redis is also not just key-value store. I "think" MongoDB is also not just key-value store. I read it as document-oriented database and I haven't had a chance to look at it closely.

You see here- all 3 popular so-called NoSQL are all not just key-value stores. This is to get the fact straight.  Cassandra supports SQL! Oh,yes, it's called CQL. I have no problem with it. I like Cassandra SQL. Let me highlight 3 CQL here-

SELECT * from People;
INSERT INTO NerdMovies (KEY, 11924)
      VALUES ('cfd66ccc-d857-4e90-b1e5-df98a3d40cd6', 'johndoe')

You can read the rest here:

All I am trying to say here is that Cassandra is a database and there is nothing wrong calling it a database!


Can you use Cassandra as a primary database to build multi-tenant SaaS applications? 
IMO, the short answer is "no". 

The long answer is that you need to use OPP (order preserving partitioner) to support the range queries for multi-tenant applications with composite keys. Datastax documentation clearly says why it's a bad idea to use OPP. Sure, there may be work-around and I don't want to design an application against work-around! Drop me an email, if you want to know more about my research here. (uday.s@comcast.net)

Side note
a) Cassandra is a 1.0 product.
     It is evolving rapidly and you will see inconsistency between pre/post 0.8.

b) Documentation is challenging. 
    Your friends are Datastax, StackOverflow and blogs. Be ready to spend time to research for information and don't get frustrated easily.

c) Cassandra is written in Java! 
    I saw inconsistency between cli and server. I have to reboot it. I saw Java exceptions on the server. Even though I spend most of the last decade @Sun/Java, I am not a fan of databases written in Java. You need to be close to metal when it comes to databases.


Apache Cassandra is a "column oriented database" written in Java and has a good promise.

Feb 4, 2012


Happy New Year - 2012!

There are lots of optimism and hope for 2012 in the tech world compared to last few years. This optimism brings new energy and new energy brings new innovations. One of the new innovation that may become mainstream in 2012 is Node.js!

What is Node.js?
A new server-side web framework using JavaScript. It's a wrapper on top of v8.

What is Node.js? (engineering definition from the web site)
Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

What is the website for Node.js?

What is v8?
It's a JavaScript Engine from Google.

Where is v8 used?
Google Chrome browser.

Who is the lead engineer for v8?
Lars Bak

Who is Lars Bak?
He was a former Sun engineer and was the lead for Hotspot JVM. If you have to count the top 10 VM engineers in the world, it's safe to say that Lars will be one of them. His engineering track record speaks for it.

Why do I think Node.js will succeed?
1) Google's V8 JavaScript engine (Lars Bak + other engineers)
2) Strong community
3) JavaScript

What is the use-case of Node.js?
Real-time communications in the web properties.

How strong is the Node.js community?
Very strong and is growing.

Is Node.js is just a framework or an ecosystem?
Node.js is a framework and it has got a strong ecosystem around it. I mean the libraries.

How big is the ecosystem?
is the answer. You can see how many libraries are already available for Node.js.

How to install programs in Node.js?
NPM (Node Package Manager)

Finally, Is Node.js competes or compliments Java?
Currently, its a complimentary framework. If this momentum continues (JavaScript for both front end and back end), Node.js may become the Tier-1 in the server-side web computing. I don't think it will replace Java but clearly lot of other frameworks like PHP, Ruby, Python, etc will take a hit with the growth of Node.

There are 2 factors that are highly influencing and favoring Node.js now.

a) Server-side JavaScript framework
The importance of JavaScript in the web computing is increasing on a daily basis and server-side JavaScript was discussed/debated for many years. Lack of good server framework made it challenge for JavaScript to succeed in the server side for long time and Node.js is the answer for it.

b) Real-time communications
In both side (consumer web & SaaS), there is a high demand for real time use cases and Node.js is becoming answer for it.

Do Yourself A Favor in 2012: Learn JavaScript!