Recently at IDRsolutions I have been working on a comparison of the various clouds and the speed in which they work with our products. While completing this task I realized that the results may be interesting for anyone who is planning to switch or move their software onto a cloud service so decided to base this article around that idea.
For this comparison I ran a Java web application (that utilizes servlets) on each cloud. This allowed me to upload a PDF file and convert that PDF file into either images or HTML using our software. I ran the test multiple times on each cloud and recorded the upload and convert times separately.
Making it fair test
Every test was carried out on the same Windows machine, using the same version of Chrome, converting the same PDF and the test was carried out around the same time on a business day. The times given are averages from multiple conversions of the same document. The first conversion results have been dismissed due to the fact that the JVM severely optimises the code during the first conversion and every conversion thereafter was significantly faster.
The code used for each cloud was slightly different as they have different infrastructures but exhibited exactly the same functionality. You can replicate the code for each test by following the relevant cloud tutorial (Image / HTML) but the vast majority of the code was the same with just a few slight tweaks so that the code ran correctly on the specific cloud.
Uploading vs Converting time
I broke the testing down into two results instead of getting an overall time because I wanted to get a better idea of where the majority of the processing time was spent. I believe the file upload time from a browser to the cloud, and the time it takes for a task to be carried out (converting the file to a different format in this case) are very important to know for potential users because you need to find a cloud that matches your current environment and workflow. Of course there are many other factors to consider when picking a cloud:
- – Price
- – How easy it is to set up and maintain
- – Which services are offered/included
- – How pretty/cool the console is
- – etc.
But in the end if you cannot work efficiently with your cloud then you will have chosen a solution that could have introduced bottlenecks in your bandwidth or created disruptions that slows the whole team down.
Like I said above the conversion timing is just the time it takes for a task to execute. This time could represent different tasks as you may not convert the document to a different format but you may store, encrypt or extract information from it. The convert functionality will just give you a general idea of how fast another cloud can perform tasks (once it has the document) in comparison to the one you are currently using or want to use.
Like any single test, there are ways we could improve it but, but overall I am happy it gives a reasonable proxy for cloud performance and comparison.
Results are given in seconds that have been rounded to 1 decimal place. The final combined table shows the average for that cloud based on the two separate tests.
Image Test Results
HTML Test Results
Combined Test Results
Take these results with a pinch of salt
This test was not carried out to say which cloud was better/more powerful/faster/cooler than any other cloud service. Originally I carried it out to give our customers’ examples of how fast our software works with each cloud’s architecture, but thought that others might have found the results useful too.
Obviously those of you who are looking for statistics to help decide on which cloud to use will not only be looking at this article. These results are just here to provide an example and any other test carried out may have different conclusions based on the software, hardware, browser, cloud structure and many different variables that were used at the time.
The upload times for the Cloud platforms are very similar and the bottleneck may be on our upload bandwidth rather than on the Cloud. If this aspect is important to you and you can potentially control your uploads, you may want to investigate this in more detail.
From my results I have concluded that with our particular software, the Oracle Java cloud is the best in terms of conversion and overall processing time. If I wanted a quicker upload time then Jelastic would be my best bet by a landslide. So that’s my findings.
Like I have said though – these are just for our software. If anyone is debating using a cloud then I would highly recommend getting the different trials and finding which works best for you. This test was completed on (mainly free) trial/lowest tier plans which were easy to sign up to, cancel and gives you a good chance to explore the different available services.
What is your experience of cloud performance?