What a wild ride it has been and continues to be at CPUsage!  This is an extremely exciting week for us. We will be performing full end to end tests on our Alpha CirrusGrid platform. I want to take a moment to reflect on some company history and provide an update on what exactly it is that our amazing engineering team has been working on and how CPUsage is driving innovation in cloud computing.

In the beginning

When our little band of brothers first set off to start CPUsage and change the world, we had a grand vision. Over the past decade due to massive innovation, exponential growth in demand for compute resources, and markedly lower cost of compute devices, there has been a huge increase in the number of compute devices such as servers, desktops/laptops, cell phones, and other smart devices being purchased. These devices are used periodically and rarely at 100% of their available capabilities, which equates to a large amount of wasted compute resources. Solving that problem became our vision and directive.

We architected and began building a platform to wrangle the world’s idle compute power based on four key pillars: scalability, flexibility, security, and the hardest to accomplish – extreme simplicity. We gleaned some major insights from this work which brought the realization that we were underselling the capabilities of this cloud management platform that we’ve now developed. The new CirrusGrid platform enables us to offer customers a true compute-as-a-utility.

What’s the problem?

There are a lot of cloud infrastructure providers and software stacks available today for public and private cloud including Amazon AWS, Rackspace, Terremark/Verizon, OpenStack, the list goes on. These providers/stacks have come a long way in recent years to offer customers flexibility to quickly spin up and clone whole environments across their respective providers. As an example let’s assume I am a customer of these services, I am building a business in the cloud and have installed my software on a cloud instance.  I need to turn this one machine into ten machines, all I have to do is submit a few API calls and voila, I have ten virtual machines with my software installed. As a customer with scalable compute needs, I have now spun up my instances on Amazon AWS or my own hardware via OpenStack and I’m good to go and can start using all these compute resources now right? Think again.

What about managing my compute workload across these machines?  Transferring data between my machines or to and from my CDN? How do I ensure a machine didn’t crash or drop offline while running one of my compute tasks or my application? As an (insert your company’s product here) supplier, Do I really have to hire a whole staff of people to manage these cloud instances for me? Can’t I just set it and forget it?

Introducing CPUsage CirrusGrid

CPUsage CirrusGrid is a generic use cloud management platform designed to run anywhere from public cloud instances (AWS, Rackspace, etc), the customer’s own private cloud, or CPUsage Dynamic Idle Compute resources. We’ve architected CirrusGrid from the ground up to work seamlessly with compute resources from any source, providing workflow automation across any cloud- public, private, or hybrid.

The CirrusGrid platform offers the following benefits:

• Infinite Scalability:  As the customer’s company scales and they submit more discreet compute jobs, our platform dynamically allocates additional devices in the background maximizing the number of compute instances required to fulfill current demand.
• Reliability:  If (and when) a cloud instance crashes or disappears, our management stack ensures the customer’s job is quickly shifted to another running instance. Jobs are not lost, leaving the customer’s data in an unknown state.
• Workload Decentralization:  If a service provider has an outage, the customer’s workflow/jobs will smoothly continue to run elsewhere.  No more stressing when Amazon’s Virginia data center drops offline for a few hours.
• Data Locality:  Cost and time savings by running compute jobs local to where the customer’s data (or customer’s data) lives.

The CirrusGrid SDK

Perhaps the most impressive part of the CirrusGrid platform is the work we have put into making high throughput computing in the cloud dead simple. Our platform SDK enables customers to deploy their application to a near-infinite numbers of devices with the same simplicity of installing the application on one device. Our sandbox environment gives customers a virtual image to deploy, test, and package their code. Once the environment is packaged, it is deployed to CirrusGrid and the customer is presented with a simple API and some sample code for submitting compute jobs to the grid. As the customer submits more tasks, more nodes will auto-provision and begin running the workload.

We’re almost there…

We’ve been building some pretty cool stuff! We’re excited to get through internal testing and start working with our customers. It’s been a long road to get here and we are thankful to have such great interest from the public on what we are working on. Now back to work…

First, I want to take the opportunity to thank each of you for your support over the past year.  You’d be amazed at what we’ve been able to accomplish with our small team of engineers and you’ve been instrumental in providing us a scalable beta platform to build out the rest of our product and test new features.  We’ve been working hard over this past year to architect, develop, and improve upon all three major vectors of our technology (which I will explain a little more below).

As some of you have noticed, we haven’t been running nearly as many jobs on our compute grid lately and therefore your accrual of points has slowed (if not completely halted).  There are a few reasons for this that I will outline below.

Demand is low while SDK is in development

Our core platform consists of three major components:

1. Cirrus Desktop Application –  Application that harnesses the unused processing power of idle computers
2. CPUsage Grid Engine – Infrastructure that manages customer workloads and distributes jobs across our grid.
3. CirrusGrid SDK – The interface to provide our customers dead simple access to the compute grid.

There’s a lot more under the hood than those three components but this is a general overview.  We have completed and have in beta the first two components and are in final stages of development of the CirrusGrid SDK.  This SDK will give our customers the ability to easily tap into the compute power that each of our compute partners offer.  We will be pushing this into beta and production over the next few months and the demand for our compute grid will quickly grow which equates to more points for you.  We already have a backlog of customers who are waiting to get their hands on the SDK so points should start accruing shortly.

More Beta devices than current demand

To efficiently test the Cirrus application, we sent out more partner invites than we had demand for compute power.  We were sending thousands of test jobs early on to rigorously test features and capabilities of the Cirrus application.  We made a decision early on to pay a distribution to our compute partners for these beta tests even though we were not yet processing jobs for or billing customers.  During the past approximately six months, testing has slowed on Cirrus as our small engineering team has been focused building out the Grid Engine and CirrusGrid SDK components of our platform.

We’ve been having a few issues with Cirrus

While much of the processing on our grid (and subsequently the accrual of points) has slowed for all of our providers in recent months, we recognize that we have a few open issues with the Cirrus application that has prevented some of our compute partners from “checking in” with our servers on a regular basis.  We have made the decision to delay fixing some of these issues to keep our resources focused on developing the CirrusGrid SDK and bringing it to beta.  We are aware of which devices are working and which are seeing issues and will be circling back around and fixing Cirrus application issues once our SDK is in beta in the coming month or so.

Hang in there!

We are extremely grateful to each of our current beta compute partners that are currently running or have run Cirrus for us in the past as it has provided very valuable insight and learnings as we build out the platform.  We have an extremely large waiting list of individuals and companies/organizations that want to contribute their compute power to CPUsage CirrusGrid.  If you already have received an invite, you are in a small group of individuals and organizations that have been accepted into our beta program and you are first in line to begin processing and receiving points once we release our SDK and open the flood gates.

Thanks again for your patience and please stay tuned for the launch of the CirrusGrid SDK in the coming months.

Sincerely,
Matt Wallington
CTO, CPUsage

To ensure that your computers are connecting to CPUsage properly, you’ll want to ensure that your firewall nor your antivirus software are blocking these applications. If you ever see a pop-up window asking if you want to grant permissions for either cirrus.exe or shelby.exe, say yes. Otherwise, CPUsage software may not be allowed to run or access the internet.

If you run Windows 7 and the built in firewall, below are instructions (source) on how to allow Cirrus.exe and Shelby.exe to communicate with the internet.

1. Open Windows Firewall by clicking the Start button , and then clicking Control Panel. In the search box, type firewall, and then click Windows Firewall.
2. In the left pane, click Allow a program or feature through Windows Firewall.
3. Click Change settings. If you’re prompted for an administrator password or confirmation, type the password or provide confirmation.
4. Select the check boxes next to cirrus.exe and shelby.exe, select the network locations you want to allow communication on, and then click OK.

If you use another popular firewall or antivirus software, look for yours below and click for more information.

Norton

McAfee

Avast!

ZoneAlarm

Karpersky

If you have any questions or run into trouble, contact our support center and we’ll assist you.

Before you leave this page to look around, consider emptying your browser cache if you’ve been to our site before.

One goal for the new site was to better communicate what CPUsage does. When you hit our homepage, you’ll see what we are all about: CPUsage makes it easy to consume massive amounts of computing resources. The site also highlights a couple higher level shifts we’ve made at CPUsage over the last year.

First, we’ve refined our product focus and rather than developing vertical specific products, we are set to launch the CirrusGrid SDK to allow any developer, anywhere, to run anything on the CPUsage grid with complete simplicity. This product is targeted at enterprise developers in industries such as: life sciences, big data, financial, mathematics, rendering, encoding, and electronic design automation. An important shift here is that we’ll help you consume compute power from any source, wether it be devices on the CPUsage grid or devices inside your own enterprise. You can read more on our Platform page and use the contact form to tell us how you’d consume massive amounts of compute power with ease.

On the Partners page, you’ll see a lot of talk about the entreprise. We know that many of you reading this post are individual computer owners with devices on our network (and desperately waiting for your points to increase), and you have nothing to worry about. We are still working hard to deliver a product that will monetize your unused compute cycles. However, our focus has been refined to target the enterprise. Truth is that there are many efficiencies for both CPUsage and the enterprises we are targeting, and focusing our partnerships on the enterprise will allow us to better serve our Platform customers. We are specifically excited to work with universities, medium size enterprises, and datacenters.

With these changes, we are very close to a full public launch of our end-to-end cloud computing solutions. Stay tuned for news about increased usage of compute power on our grid and the launch of our CirrusGrid SDK for utility computing. We hope you enjoy the new site and find it to be a nice upgrade of both content and design. Thanks for your continued support of CPUsage.

The goal for CPUsage from day one has always been to change the dynamic of cloud computing. We were (and still are) frustrated with the idea that to consume massive compute power, you must focus on complex technical and business problems common with existing cloud solutions. These problems include determining which compute instances make sense from both a technical and business standpoint.  Questions such as “How many instances do I need?” “How should these instances be provisioned?” “How do I provision for peak demand but save costs when under-utilizing the compute power I’ve purchased?” Businesses (such as fellow Portland company Cloudability) have popped up to solve some of the issues around tracking costs. Software companies have emerged to create solutions to help you deploy environments across multiple devices or instances (Puppet, Chef).

For industries that have a massive need for scalable compute power to solve real world problems, like life sciences, electronic design automation (EDA), or financial analysis, are you comfortable spending your day managing cloud computing instances? Figuring out how to deploy your workload across instances? With figuring out how to deal with a stuck instance, or worse yet an instance that disappears or drops offline? Do you want to hire an entire team to manage the cloud services you deploy your work to? I thought the purpose of cloud computing was to eliminate the need to manage servers and infrastructure. Wouldn’t you rather spend your day focused on providing solutions to the fundamental problems that are core to your business rather than focusing on managing cloud instances, deploying software across thousands of nodes, and splitting compute workloads between the instances?

The CPUsage Advantage: Compute-as-a-Utility

CPUsage has built a platform that enables anyone with high-throughput computing needs to tap into the power of thousands of high performance nodes with the ease of managing a single instance.

CPUsage provides a compute-as-a-utility similar to electricity.  Plug in and consume as much as you need, at very low cost, and don’t worry about sourcing additional power. Don’t worry about how many instances you need or whether there will be enough compute power available. Don’t worry about what types of instances you should order to save costs, or how to keep the instances running only when you need them.

Below are some of the great features CPUsage provides to not only make your life easier but save major costs.

• We handle distributing the workload to the most capable nodes for your task.
• Our system manages the number of nodes required to fulfill your workload in the quickest time possible.
• We take care of automatically redeploying your workload if a node fails or goes offline.
• You pay only based on exact processing time.  No more spinning up and down instances to save costs.
• We deploy to additional systems as fast as you push tasks at us.  No more queuing tasks on instances and waiting for prior work to complete.  We can scale near infinitely to meet your demands.

The CirrusGrid SDK

We are excited to announce that we are in the final stages of developing our alpha SDK.  Our target for releasing the SDK to public is late Q1 or early Q2 2013.  Our SDK provides dead simple access to our scalable compute grid for any task that runs on the Linux operating system.  Deploying your workload to CirrusGrid is as simple as installing your environment on a single instance and providing a simple set of instructions on how to run the task on that node.  The CirrusGrid SDK then validates your environment and deploys it to our scalable compute grid.  Nodes get provisioned on the fly ensuring we can scale to as many nodes as you will ever need automatically.  The CirrusGrid SDK provides simple examples for deploying your workload.

What could take months to develop and deploy on standard infrastructure providers will take hours, if not minutes to deploy to CirrusGrid.  Once your software is configured, validated, and deployed we provide a simple method for submitting your workload to our scalable grid.

Stay tuned for more updates and feel free to drop us a line if you have any questions or would like more information.

GridVid.me is an API based cloud video encoding service ideal for Online Video Platforms like Vimeo, Viddler, and Vuier. (And no, your company doesn’t have to have a name starting with the letter “V” to use it). Its fast, highly scalable, easy to use, and the best price in the industry.

The launch was covered by some great outlets, including TechCrunch:

So how can it afford to offer encoding at a much lower price than competitors? Its secret is that unlike cloud encoding vendors, GridVid.Me doesn’t pay Amazon or other cloud computing providers for its compute cycles. That’s because GridVid.Me is part of CPUsage, a firm that leverages the unused compute cycles of idle PCs for profit. Kind of like a for-profit SETI@Home, CPUsage gets users to install a plugin on their PCs which runs in the background while a computer is dormant or using a limited amount of processing power. It can run various commercial applications, for which CPUsage pays a fraction of the cost of regular AWS services.

This is just the first of many industry-disrupting products so stayed tuned for more! In the meantime, check out GridVid.me at http://gridvid.me, follow on Twitter (@GridVidme) and like on Facebook (/GridVidme).

These contests detract because we put far too much into them. Energy and mind-space that could be spent on our company, our products, customers, employees, and ourselves. Instead of asking our customers to do something for us, we should ask our customers what we can do for them (not an intentional JFK reference). Frankly, I am now embarrassed that I asked for your help while many of you are still waiting patiently to become a CPUsage customer or partner.

The straw that broke the camels back for us was the contest we were recently finalists in called Jumpstart. It was put on by two great companies and the prize was huge: $100,000 worth of digital marketing services. We applied on a whim after a friend sent it to us and soon after, we learned we were selected as a finalist by a panel of judges. All that remained was to get the most votes….how hard could that be?

So we put it out to our company network and the personal networks of all 6 of us that work here. We begged and pleaded…. “please, please vote for us!” When you voted, you had to provide the contest organizer with your full name, email address, and your company/work info. CPUsage (and the contests organizers) asked a lot of you for your vote.

However, we asked you to do that for naught. See, we were the 4th smallest of the 5 finalists by many measures. Fewer customers or online fans. So what happened? We got blown out by companies with a larger footprint than us. We rallied fewer than 200 votes and if I recall correctly, the winning company had around 2,000. This contest was simply a measure of who was bigger….who had more ears listening to them. Not who was most in need or worthy of the prize.

In fact, that opposite argument could be made. The contestant with the fewest votes is likely the company in most need of digital marketing services! So that is what I want to see someone do, a contest were the loser is the winner.

So are these contests worth it? Absolutely not….unless of course you are the winner. For CPUsage though, we are going to stay away and put 100% of our focus on building a great product and changing the world. You can have your contests, we are building something bigger.

What Happened

We wanted to send an update to the 700+ partners that provide CPUsage with idle compute power. I used a new web based email drafting tool we built into our admin portal. We use SendGrid to send emails, ensuring they get delivered properly and to make sure you have the option to unsubscribe easily.

What we failed to do is make sure emails were sent to recipients using the BCC field. When I clicked send on our first email to our partners, we mistakenly shared everyone’s email address in the “To” field. It is a terrible, amateur, and stupid error.

Why It Happened

This happened for a pretty simple reason: we didn’t properly test our code. Sure, we could have also avoided this by properly implementing SendGrid in the first place, but that is what testing is for….as last line of defense. No excuses, we screwed up. Properly testing would have caught this. We were too eager to move on to the next thing on our to-do list.

What We are Doing About It

We hope you know by now that they first thing we did was take responsibility for our screw-up. We are owning it.

Next up is fixing the code….and properly testing it! The code was fixed within 5 minutes of us recognizing the error. We’ve been testing since then and will do so for another hour or two tonight. We have tested in a development environment, a “test” environment, and in an “beta” environment. Next up is to send a message from the production environment, to a handful of select recipients. If that passes muster, we’ll send you all another email to prove that the problem is fixed.

After the code is fixed and tested, we’ll be reviewing our testing procedures, specifically to put a check and balance system in place to ensure this doesn’t happen again.

We’ll also be crediting the accounts of all active CPUsage partners. This probably isn’t necessary, and it could be looked at as a “here is some money, don’t be mad at us” kind of thing. Really though, its putting our money where our mouth is. It is a way to show you that CPUsage is committed to doing the right thing, and we are willing to fine ourselves if we screw up. We also figure that with the lack of work units lately, you wouldn’t mind seeing a few more ponts in your account anyway.

—–

From here, its our job to earn back your trust. You have our word that we’ll do everything in our power to make sure a screw-up like this doesn’t happen again. Truth is, we’ll probably screw up in the future…but we are committed to learning from this mistake and not letting the same ones happen again. If you’d ever like to talk to us about this screw up, or any other issues, please feel free to contact us. Our office phone number is 503-985-8278. Our company email is contact@cpusage.com and my email address is jeff@cpusage.com. If you have any suggestions for us, please feel free to leave them in the comments.

Thank you for your forgiveness, understanding, and continued support.

Jeff Martens & the CPUsage Team

We’ve pulled together the power of thousands (scalable to hundreds of thousands) of idle computers ready to begin working on your tasks.  We have built an Infrastructure-as-a-Service enabling customers with parallel workloads to deploy their tasks to our grid maximizing compute efficiency and minimizing the costs and time it takes to process tasks.  Some examples are in video encoding, graphic rendering, life sciences, algorithmic trading, and financial analysis (Monte Carlo simulations).

We work directly with customers looking for compute power and with companies looking to provide a Software-as-a-Service for customers with scalable needs.  On average we save over 50% of the costs of traditional cloud infrastructure providers such as Amazon EC2 and Google GCE and work with customers to provide a much easier to use scalable environment.

Please contact us today to learn more or schedule a free consultation.

HTTP Live Streaming (HLS)
We now support encoding and segmenting videos into chunks for HLS.  This enables customers to break a stream into small HTTP-based file downloads.  This feature generates the required .m3u8 playlist file and .ts video/audio chunks required for Apple HLS.  For an example use case, visit our GridVid documentation.  For more information on HTTP Live Streaming visit the HLS Wikipedia page.

FTP File Transfer
In addition to transferring input and output files between Amazon S3, we have now included support for FTP.  FTP enables customers who store files locally (or do not use Amazon S3) to utilize CPUsage for Encoding.  For documentation please visit the GridVid documentation.