Transitioning MAM to the Cloud | Imagen Blog

After a 2-year sabbatical, I am delighted to be back at Imagen, even more so that the market is coming towards cloud MAM like a tidal wave.

There seems to be a perfect storm of need and affordability.  Remote working in the face of COVID has driven some demand, but more likely just accelerated the inevitable.

For our clients and prospects, there are two main drivers that I see; firstly there are more “mouths to feed”, that is more OTT platforms, more sponsors, more partners who need to select content directly from the MAM that traditionally would be prescribed and pushed to them, secondly, equipment purchased in the 2013 – 2015 timeframe is end-of-life and has not been replaced, such that it is now urgent.

Those who resisted upgrading and replacing their on-premises MAMs and libraries are now being rewarded for that decision as the cloud economics have, somewhat abruptly, become very favorable.

I have been reflecting on these market dynamics, reviewing the evolution and thought to share some observations, hoping to prompt a discussion.

Cloud MAM origins

Imagen had their first “mobile” product demo back in the 2000's. Prior to this it was necessary to come to the Cambridge office to get behind the workings of the system, until we managed to Ghost a version onto mini-server weighing 9.6kg which could fit into a medium sized wheely suitcase to take to exhibitions and client meetings.

Albeit progress, it was still prohibitively expensive to make custom demos and with issues of Windows server licenses, the Imagen engineers tried out deploying a version of our software on the then fledgling Microsoft Azure IaaS platform. Not only did it work, but it became immediately obvious that cloud was the future for the product itself.

Imagen secured our first cloud customer shortly afterwards (who are still very much with us today), but the rate of adoption was frustratingly slow.

Objections to cloud

As with anything new, the immediate response was “is it safe, does it work and how much does it cost?”

Safe started off by meaning “will my data be secure”. Advocates would say, “well you put your money in a bank, why not put your content in a cloud?” Nay-sayers were scaremongering about the US Patriot Act and the farcical notion that, if you store your media with a US company, regardless of where it is hosted geographically, you are giving the US government the right of ownership.

Connectivity and cost were legitimate constraints, especially for broadcast file types and large libraries.

Does it work and is it safe?

Simply, yes; it works, and what is more, connectivity is abundant in most places now, but you do need to be serious about security, make sure your suppliers meet standards such as DPP Committed to Security or ISO-27001 and just look under the hood and make sure replications and back-ups observe the best practices of checksums and data-integrity.

How much does it cost?

In 2013/14 I was regularly building quotes for clients seeing an on-premises LTO-6 library, each of which I retained and reviewed for this article.
I have an archive of these quotation for all the major HSMs complete with an LTO library, a cache of spinning disc and servers.

For the ease of comparison, I have selected the lowest cost provider then and calculated a per TB per month rate based on a 5 year-year life span.
Of these, the best price for 1 PB usable storage worked out at $4.44 per TB per month for a single copy. Of course, any underutilised capacity would increase this average cost per TB and you have to bear in mind the need for air-conditioned server rooms, connectivity etc, but let’s use that as a base line.

At the same time, a TB of Microsoft Azure geo-redundant storage (GRS) was $97.28 per TB per month, thus a fully cloud hosted MAM solution would never have passed CFO approval.
Looking at cloud in 2021, that same GRS is $36.80 per TB per month, that is 62% lower than 2014 thanks to price wars between Azure and Amazon plus new entrants.
However, another major advance came in the form of tiered storage, AWS Glacier of course, but also the more practical cool storage tiers (which, at the time of writing, Azure Cool is priced at $20.48 per TB per month).

Cold archive storage is currently quoted at $2.86 per TB per month for AWS Glacier Deep Archive at $1.84; seemingly very attractive, but…...

…there are a whole host of unitised costs, bandwidth and API operations, which vary widely between different storage providers and tiers and are near impossible for any mortal being to predict and model. Simple bandwidth consumption alone could deliver you a real shock at end-of-month “true-up”. For instance, 1 TB of internet egress from Azure Hot is $89.30, and $100.30 from Cool while Cold cloud, such as Azure Archive and AWS Glacier charge astronomical rehydration fees, depending on when you want your data back.

Enter the new players, notably Wasabi which is perfect for this large media library use case. Hot storage with no read, write nor egress costs at $5.99 per TB per month…..could be the start of another price war?

Cost examples

Now let’s get practical and try and compare some apples. I trust that, from your interest thus far, you are a sensible organisation, aiming to observe best practices in video management, you will be looking to adopt the 3:2:1 principle, that is 3 copies on 2 different media with one off-site.

Here are some on-premise and cloud scenarios, each satisfying your 3-2-1 ambitions with high-availability and 2 back-ups. Pricing is normalised to a per TB per month for ease of comparison.

Scenario 1 – on-premises, with hot-disc storage and replicating LTO libraries two geographically distributed locations – hardware, software and support included, premises, electricity and networking excluded.

Scenario 2 – cloud hot storage with 2 cloud archive copies (write once, only restore in an absolute disaster) based on pay-as-you-go pricing for 1TB.

But this is just dumb storage. You need a full MAM and database replication, access controls and workflow tools to be operationally efficient and secure.

What is going to manage it?

The traditional HSM and MAM providers have options for hybrid storage, with policy-based life-cycle management to move content around the various storage tiers. This could help you eke out some dying breath from your old hardware but upgrading legacy tech to simply arrive at a hybrid state is undoubtedly a false economy.
With any luck, you or your predecessors will have thought of this at the beginning, heeded the good word of the DPP Guide to Digital Archiving and have a well-documented Exit Strategy planned for this eventuality. Though this being the real-world, that is unlikely.

You will need to rehydrate your media from LTO to disc and egress to the cloud, coupled with all existing metadata from your legacy MAM in a format your cloud MAM will understand and be able to associate back with the media.

This sheer concept can be overwhelming, and, as I am learning from the various projects I am involved in, incumbent suppliers are always very reluctant to provide support for such an exit.

CAPEX and OPEX

Moving to a fully cloud MAM offers a true opex model which is more attractive to most finance directors, but this comes with the risk of variability, whereas hardware, despite being ugly for cash-flow, is predictable as a fixed asset amortised over a 3 or 5-years period.

In the vast majority of cases, with the right mix providers, cloud is more economical, scalable, accessible and future-proofed.

In the 100TB to 1PB range specifically, given the right mix of providers is nearly 1/3^rd the cost of running an on-premises equivalent in terms of hard-costs, payment models, access, staffing and support.

The last remaining barriers to a full cloud migration are:

• sunk costs in existing equipment
• fear or an inability to model consumption
• proprietary lock-ins with incumbent providers inhibiting the move

Why Imagen?

Notwithstanding the elegant user interface, sophistication of metadata and media management, security and access, Imagen offer you a fully managed 3-2-1 compliant MAM solution.

Moreover:

• Imagen do not charge a set-up fee.
• Imagen do not charge for media ingest from existing systems.
• Imagen do not charge any unitised costs above your forecast future storage consumption.
• Imagen do not charge you for your data retrieval.
• Imagen offer you an annual break-clauses so there are no long-term lock-ins.

Finally, Imagen commit to supporting a future exit without charges; giving you and your successors peace-of-mind, whilst keeping our investment firmly focused on advancing the platform with your needs such that you would never want to leave.