Georgia Ingham

Georgia is a Java Developer at IDRSolutions and mainly works on maintaining and improving the core product code. Her hobbies include reading, completing puzzle books and cycling.

Performance Comparison of 5 Cloud Platforms

3 min read

Cloud Performance
Comparing various Cloud Performances

Recently at IDRsolutions I have been working on a comparison of the various clouds and the speed in which they work with our products. While completing this task I realized that the results may be interesting for anyone who is planning to switch or move their software onto a cloud service so decided to base this article around that idea.

For this comparison I ran a Java web application (that utilizes servlets) on each cloud. This allowed me to upload a PDF file and convert that PDF file into either images or HTML using our software. I ran the test multiple times on each cloud and recorded the upload and convert times separately.

Making it fair test

Every test was carried out on the same Windows machine, using the same version of Chrome, converting the same PDF and the test was carried out around the same time on a business day. The times given are averages from multiple conversions of the same document. The first conversion results have been dismissed due to the fact that the JVM severely optimises the code during the first conversion and every conversion thereafter was significantly faster.

The code used for each cloud was slightly different as they have different infrastructures but exhibited exactly the same functionality. You can replicate the code for each test by following the relevant cloud tutorial (Image / HTML) but the vast majority of the code was the same with just a few slight tweaks so that the code ran correctly on the specific cloud.

Uploading vs Converting time

I broke the testing down into two results instead of getting an overall time because I wanted to get a better idea of where the majority of the processing time was spent. I believe the file upload time from a browser to the cloud, and the time it takes for a task to be carried out (converting the file to a different format in this case) are very important to know for potential users because you need to find a cloud that matches your current environment and workflow. Of course there are many other factors to consider when picking a cloud:

  • – Price
  • – How easy it is to set up and maintain
  • – Which services are offered/included
  • – How pretty/cool the console is
  • – etc.

But in the end if you cannot work efficiently with your cloud then you will have chosen a solution that could have introduced bottlenecks in your bandwidth or created disruptions that slows the whole team down.

Like I said above the conversion timing is just the time it takes for a task to execute. This time could represent different tasks as you may not convert the document to a different format but you may store, encrypt or extract information from it. The convert functionality will just give you a general idea of how fast another cloud can perform tasks (once it has the document) in comparison to the one you are currently using or want to use.

Like any single test, there are ways we could improve it but, but overall I am happy it gives a reasonable proxy for cloud performance and comparison.

The Results

Results are given in seconds that have been rounded to 1 decimal place. The final combined table shows the average for that cloud based on the two separate tests.

Image Test Results

Upload
Convert
Total
Oracle Java Cloud
4.1
1.2
5.3
Jelastic LayerShift
1.3
4.6
5.9
Red Hat OpenShift
4.3
6.2
10.5
Microsoft Azure
4.6
6.7
11.3
Amazon AWS
5.2
6.5
11.7

 

HTML Test Results

Upload
Convert
Total
Oracle Java Cloud
4.1
1.3
5.4
Amazon AWS
4.0
3.6
7.6
Jelastic LayerShift
1.0
7.4
8.4
Red Hat OpenShift
4.1
5.0
9.1
Microsoft Azure
5.7
10.0
15.7

 

Combined Test Results

Upload
Convert
Total
Oracle Java Cloud
4.1
1.3
5.4
Jelastic LayerShift
1.2
6.0
7.2
Amazon AWS
4.6
5.1
9.7
Red Hat OpenShift
4.2
5.6
9.8
Microsoft Azure
5.2
8.4
13.6

 

Take these results with a pinch of salt

This test was not carried out to say which cloud was better/more powerful/faster/cooler than any other cloud service. Originally I carried it out to give our customers’ examples of how fast our software works with each cloud’s architecture, but thought that others might have found the results useful too.

Obviously those of you who are looking for statistics to help decide on which cloud to use will not only be looking at this article. These results are just here to provide an example and any other test carried out may have different conclusions based on the software, hardware, browser, cloud structure and many different variables that were used at the time.

The upload times for the Cloud platforms are very similar and the bottleneck may be on our upload bandwidth rather than on the Cloud. If this aspect is important to you and you can potentially control your uploads, you may want to investigate this in more detail.

The conclusion

From my results I have concluded that with our particular software, the Oracle Java cloud is the best in terms of conversion and overall processing time. If I wanted a quicker upload time then Jelastic would be my best bet by a landslide. So that’s my findings.

Like I have said though – these are just for our software. If anyone is debating using a cloud then I would highly recommend getting the different trials and finding which works best for you. This test was completed on (mainly free) trial/lowest tier plans which were easy to sign up to, cancel and gives you a good chance to explore the different available services.

What is your experience of cloud performance?

If you’re a first-time reader, or simply want to be notified when we post new articles and updates, you can keep up to date by social media (TwitterFacebook and Google+) or the  Blog RSS.

Georgia Ingham

Georgia is a Java Developer at IDRSolutions and mainly works on maintaining and improving the core product code. Her hobbies include reading, completing puzzle books and cycling.

3 Replies to “Performance Comparison of 5 Cloud Platforms”

  1. Thanks for sharing.

    What cloudlet settings (limit) did you use on Layershift Jelastic? It would be interesting to know if you maxed out the resources (RAM/CPU) during the test, or if there was some other bottleneck.

    1. For these tests I used the default configurations for each cloud. I was using the Jelastic trial when these timings were recorded so it would have been the default settings for a Java application on a trial subscription. If there is further interest we are considering running further articles looking at each cloud and optimizations in more detail.

      1. Not exactly about optimisations. The cloudlet settings are effectively the “instance size”. The default offered to you during the creation wizard is only 6 cloudlets (1.2GHz CPU, 768MB RAM). On trial accounts you have the option to set this up to 64 cloudlets (12.8GHz).

        With 10x increase in CPU power I’m pretty sure the convert step will run much faster on our platform. Combined with the much faster upload times I think you’d have a much different winner.

        That’s without setting anything differently within the application or server level – just moving a slider as you create the environment. You can try this on any trial account, but feel free to contact me if you want to discuss the results in more detail.

Leave a Reply

Your email address will not be published. Required fields are marked *