CrashPlan throws in the towel … for home users

Today CrashPlan, my current online backup provider, announced on Facebook of all places, that they threw in the towel, and will no longer provide service to home users. The backlash was heated, and I found the CEO’s video message on the blog post rather condescending.

I’ve been a long time user of online backup providers, and many have thrown in the towel, especially when free file sync from Google and Microsoft offers ever expanding capabilities and more and more free storage. Eventually even the cheapest backup storage implementation becomes expensive, when compared to a cloud provider, and not profitable as a primary business.

I’ve been using CrashPlan’s unlimited home plan for quite some time now, they were one of a few, today none, that were reasonably priced, allowed unlimited storage, and supported server class OS’s. But, I could sense the writing was on the wall; they split the home and business Facebook account, they split the website, the home support site has not seen activity in ages, they made major improvements to the enterprise backup agent, switching to a much leaner and faster C++ agent, while the home agent remained the old Java app with its many shortcomings, and there were some vague rumors on the street of a home business selloff attempt.

The transition offered a free switch to the small business plan, for the remaining duration of the home subscription, plus 3 months, and then a 75% discount on next year’s plan. For my account, this means free CrashPlan Pro until 12 June 2018, then $2.50 per month until 12 June 2019, and then $10.00 per month.

I’ve switched to the Pro plan, as they promised the agent updated itself, going from the old Java to the new C++ agent, the already backed up data was retained without needing to backup again, and all seems well, for now…

CrashPlan Memory Utilization

I’ve been using CrashPlan as an online backup solution for quite some time, and it works really well.

I like the fact that I can subscribe to the consumer plan, with almost 3.5TB of data backed up, and that the backup client installs on a server OS. Many of the other “unlimited” backup providers I tested have restrictions in place that makes such a setup impossible.

CrashPlan sends email notifications about backup status, and I noticed that something was wrong with the backup:
CrashPlan.Email

I logged onto the machine, opened the main UI, and after a few seconds the UI just closed. opened it again, same thing, after about 15s the UI closed.

My initial thoughts were that it is a crash, but on attaching a debugger, the exit call stack showed that the process was cleanly terminated after receiving a signal.

On looking at the NT eventlog I could see that the service was restarting about every 15s:

The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.
The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.
The CrashPlan Backup Service service entered the stopped state.
The CrashPlan Backup Service service entered the running state.

The service wasn’t crashing, it was externally being stopped and restarted. I looked in the CrashPlan directory, and I found several log files with a naming like restart_1342296082496.log. The contents of these files looked like this:

Sat 07/14/2012 13:01:22.53 : "C:\Program Files\CrashPlan\bin\restart.bat"
ECHO is off.
Sat 07/14/2012 13:01:22.53 : APP_BASE_NAME=CrashPlan
Sat 07/14/2012 13:01:22.53 : APP_DIR=C:\Program Files\CrashPlan
ECHO is off.
Sat 07/14/2012 13:01:22.53 : Stopping CrashPlanService
The CrashPlan Backup Service service is stopping.
The CrashPlan Backup Service service was stopped successfully.

Sat 07/14/2012 13:01:25.05 : Sleeing 15 seconds...

Pinging 127.0.0.1 with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Sat 07/14/2012 13:01:39.08 : Starting CrashPlanService

The CrashPlan Backup Service service was started successfully.

ECHO is off.
Sat 07/14/2012 13:01:39.13 : Exiting...

I looked for a newer version, but 3.2.1 was the latest version. I logged a support ticket with CrashPlan, but I continued my investigation. I found a log file service.log.0, several MB in size, and inside it I found this:

[07.14.12 12:32:39.480 ERROR   QPub-BackupMgr       backup42.service.backup.BackupController] OutOfMemoryError occurred...RESTARTING! message=OutOfMemoryError in BackupQueue!

So it seems that the service is running out of memory. I now had a few good keywords to search on, and I found this post of a user with the same problem. At about the same time I received a reply from CrashPlan support, not bad for weekend service, with the same solution.

The CrashPlan backup service and desktop applications are Java apps, and as such the maximum amount of memory they use are capped by configuration. I have had similar problems with other memory hungry Java apps, like Jaikoz, that simply fail unless you increase the memory limit.

To fix the problem, shutdown the service, open the CrashPlanService.ini file in the program directory, and increase the maximum memory utilization parameter to 2GB, the default is 512MB, and restart the service:

Virtual Machine Parameters=-Xrs -Xms15M –Xmx2048M

After upping the memory all seemed well, and the service has been running for more than a day. But, I wanted to know just how much memory is CrashPlan using, and it turns out to be insane.

Here are the current stats for the amount of data I backup, as well as the resource utilization by the backup service and desktop app:

CrashPlan.Size
CrashPlan.Memory.Desktop
CrashPlan.Memory.Service

As you can see, the desktop app’s peak private bytes exceed 250MB, and the service exceeds 1.3GB, that’s right 1.3GB of memory!

Those numbers are simply outrageous.

Unlimited online backup providers becoming extinct

I just received an email from ElephantDrive informing me that my legacy unlimited storage account will be terminated in 30 days, and that I must select a new plan.

In July 2009 ElephantDrive announced that they are no longer offering their $100 per year unlimited storage plan. ElephantDrive is now offering a $200 per year for 500GB plan.
In February 2011 Mozy announced that they are no longer offering their $55 per year unlimited storage plan. Mozy is now offering a $120 per year for 125GB plan.
In February 2011 Trend Micro SafeSync announced that they are bandwidth throttling large accounts. In March 2011 they announced that they are no longer offering their $35 per year unlimited storage plan. SafeSync is now offering a $150 per year for 150GB plan.
Carbonite offers a $55 per year for unlimited storage plan, but they are bandwidth throttling accounts over 35GB to 512Kbps and accounts over 200GB to 100Kbps access speeds.
AVG LiveKive offers a $80 per year for unlimited storage plan, but the terms of service defines unlimited as 500GB.
BackBlaze offers a $60 per year for unlimited storage plan.
CrashPlan offers a $50 per year for unlimited storage plan.
Neither BackBlaze nor CrashPlan supports their unlimited plan on server class machines.

I currently have 2.1TB of data backed up online with ElephantDrive running on my Windows Server 2008 R2 machine. Needles to say, none of their new plans are affordable for that amount of storage. I either need to significantly trim down what I backup, or I need to find a new unlimited storage provider, that also allows installs on Windows Server.
For now, I’m uninstalling ElephantDrive.

[Update]
CrashPlan’s new v3 software installs and runs fine on Windows Server 2008 R2, and I have switched to using CrashPlan for my backup needs.

Here is an example snippet of the status emails I receive from CrashPlan:

Source → Target Selected Files Backed
Up %
Last
Connected
Last
Backup
VM-STORAGE → CrashPlan Central 2.1TB ↑1KB 423k 0 100.0% 2.5 hrs 4.3 hrs

Trend Micro SafeSync, neat, but unreliable

I wanted to write about Trend Micro SafeSync, but it reminded me of my Streamload experience, and I ended up writing this instead. This time I am really going to write about SafeSync.

SafeSync is another online backup and sync and share application. Actually, they offer both online storage through a mapped drive, and syncing folders online, this makes it unique compared to many existing offerings.

I have used almost all online backup and sync and share type applications out there, my favorite remains DropBox. SafeSync used to be Humyo, before being acquired by Trend. I have used Humyo when they were in Beta, it was just ok, but between then and now their product seem to have come a long way.

Of all the online backup and sync and share applications, a few things remain constant;
Free is unsustainable, somebody has to pay for the staff, the bandwidth, the disks, and the infrastructure. These vendors are running on venture capital, waiting for acquisition, for paid customers, for indirect monetization, or failure.
Unlimited storage is unfeasible, the increased home bandwidth capacity makes it easy to upload Terabytes of data, and we are back at the cost factor.
Usability and coolness is critical, especially usability on mobile devices, and coolness on web frontends.
Reliability is critical, and this brings me back to SafeSync.

SafeSync offers many things common to many other backup or sync and share providers, but three things stood out; they offer unlimited storage, they offer data access using WebDAV, and the web frontend allows convenient access to pictures and music.

The product is offered as a yearly service, listed as $59.99 on the Trend eStore, or $35.95 on the Trend US product page, weird. Regardless, when you add the product to the cart, the cost is $35.95.

I installed the software on three systems, two running Windows 7 Ultimate x64, and one running Windows Server 2008 R2. The install creates a new drive that is mapped to the online storage, and a user session application that can be used to sync a local folder to the online storage.

Here are some screenshots:
SafeSync.Config.Folders
SafeSync.Config.Connection
SafeSync.Status

The web frontend really reminds me of Streamload:
SafeSync.Online

A very neat feature is WebDAV access to the storage. This means that you can access the data using any WebDAV client, and there is no need to install the SafeSync client software. Here is a Trend KB for details, basically you connect to “dav.trendmicro.safesync.com” using your SafeSync credentials.

You can use the built in Windows WebDAV client to access the storage, but you have to make a registry change, else you will get a "the folder you entered does not appear to be be valid" error. After you make the change reboot, or just restart the WebClient service. See this Microsoft KB for details:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\WebClient\Parameters]
"BasicAuthLevel"=dword:00000002

Open explorer and map a network drive to “\\dav.trendmicro.safesync.com”:
WebDAV.Windows

Here are some explorer screenshots of a mapped drive using SafeSync, Windows, and WebDrive:
SafeSync.Cross
Windows.Cross
WebDrive.Cross

So this all sounds great, well, not so great, the client application has serious stability issues.

On two machines, every time I logout of Windows, Windows reports that SafeSync is not responding, after a minute or so, Windows eventually logs out.
On one machine, every time I logout, Windows paints the logging out screen, and never completes, requiring a power cycle.

SafeSync interferes with applications that are accessing files in a shared folder. It appears that SafeSync notices a file modification, then opens the file, and does not allow other applications access to the file. As an example, I create backups of my CD collection using dbPoweramp, and I shared the output folder in SafeSync. While dbPoweramp is still using the files, SafeSync opens the file and dbPoweramp fails. This is not a problem with dbPoweramp, and other sync applications, like DropBox, work just fine in the same situation.
dBPoweramp.Error.Writing

Adobe PhotoShop CS4 x64 crashes every time I open an image that is located in a shared folder. The crash is caused by the SafeSync explorer shell extension.
FAULTING_IP:
HrfsShellExtension!DllUnregisterServer+202ef

If a sync is in progress, and the machine goes to sleep, then later wakes up, SafeSync does not reconnect, instead it reports that the server is unavailable. In order to resolve this you have to logout and log back in.

I added a folder to sync, this folder was very large, the status window indicated it would take several days to complete, I wanted to remove the mapping. On clicking the remove button, I received this funny error message, “Unexpected and unknown error, it is possible a logical error”. The only way to stop the sync was to uninstall.
SafeSync.Logical.Error

SafeSync crashed while uninstalling.
SafeSync.Uninstall.Crash

Lastly, the website does not display any file extensions.

I have not quite given up on the service, just the client software, it is simply too buggy. I am still using the online storage through a WebDrive mapped drive.
But, I still do not believe that unlimited storage is a sustainable business practice, and I would not be surprised if SafeSync limits storage, dramatically increases pricing, or is terminated.

Will your online backup or sync and share provider survive?

I have been using online backup for many years. I won’t call myself an expert, experienced maybe, but I have tried almost all online backup and sync and share services. My biggest concern is still the longevity and trustworthiness of the provider.

My biggest disappointment is definitely Streamload. Streamload was a great service, they provided unlimited storage, had a client up-loader, supported FTP access, and had a great web frontend. And this was in 2005, long before many other online storage providers existed, and nobody else offered unlimited storage. The concept was great, take any of your online files, and present them in a web friendly way, e.g. play your backed up music collection from your browser, watch your backed up movies from your browser.

Unfortunately things did not go so well. Over time Streamload became MediaMax became TheLinkUp, none survived, but a spinoff Nirvanix did. Unlimited storage became limited storage, FTP access was revoked, free accounts became paid only accounts or loose your data. The final death-blow came when user account data was “accidentally” deleted, about the same time as MediaMax filed for bankruptcy, leaving thousands of angry users with no access to their data. Needless to say I was less than happy, but no point in reliving the past, you can read about the saga here, here, here, here, and here.

By the time MediaMax died, I had close to 1.3TB of data online. Yes, that’s a lot, but it was so easy to add without concern for size (MediaMax no longer offered unlimited storage, but unlimited accounts were grandfathered in), whenever I made a local backup, I just uploaded it online, and never deleted old backups.

My point is, a backup is only as good as its ability to restore, and what good is your backup, if you can’t access your data?