Role of Amazon behind the success of Netflix

How Netflix start their journey?

Netflix was first founded in August of 1997 by two serial entrepreneurs, Marc Randolph and Reed Hastings. The company began out in Scotts Valley, California, and has grown to become one of the world’s leading internet entertainment platforms.

When it first opened, Netflix was purely a movie rental service. Users ordered movies on the Netflix website and received DVDs in the post. When they finished, they would post them back to Netflix in the envelopes provided. At the time, this was seen as a boon to those who did not have a video rental store nearby

Why Netflix move to cloud?

Netflix was originally a DVD shipping business where they would send out DVDs of your chosen programs to you. This was going well until 2008 where they experienced a major database loss and for 3 days could not ship out any DVDs to their customers. That was when the senior management at Netflix realised that they had to shift from continuous vertical scaling which leads to single points of failure to a more reliable and scalable horizontal scaling system. They chose Amazon Web Services despite having Amazon as a competitor (Amazon has their own streaming service known as Amazon Prime) because AWS provided them with the greatest scaling capabilities and the biggest set of available features. It took 7 years of migration for Netflix to shut down their last remaining data centers and move completely to the cloud.

Moving to the cloud has allowed Netflix to keep its existing members well engaged with overall viewing growing exponentially.

Netflix itself has continued to evolve rapidly by using many new features and relying on ever-growing volumes of data. Supporting this fast growth would not be possible earlier using their own in-house data center. Netflix could not have racked the servers fast enough to support their own growth. While Cloud brings elasticity, which allows Netflix to add thousands of virtual servers and petabytes of storage within minutes which makes the whole process easier.

As of January 2016, Netflix has expanded into 130 new countries. It uses multiple AWS Cloud regions which are spread all over the world to create a better and more enjoyable streaming experience for Netflix members wherever they are.

Netflix relies on Cloud for all its scalability, computing, and storage needs (not only video streaming) — Netflix business logic, distributed databases, big data processing, analytics, recommendations, transcoding, and hundreds of other functions that are used by Netflix all go through their Cloud infrastructure. Netflix also has its own Content Delivery Network (CDN) known as Netflix Open Connect which is used to deliver videos globally in an efficient manner.

When Netflix was using their own data center, it faced a lot of outages. Cloud Computing is not perfect either, even though Netflix has hit some rough patches in the cloud, a steady increase in the overall availability has been noticed. Failures are ultimately unavoidable in any large-scale distribution system, even a cloud one. However, a Cloud-based system allows you to create redundancy measures while become quite helpful. Cloud Computing has made it possible to survive failures without impacting the member experience.

Netflix did not shift to cloud for cost reduction reasons, but Netflix’s cloud costs ended up being a fraction of their cost which was a pleasant surprise. This was due to the elasticity factor of cloud computing, enabling Netflix to continuously optimize instances to grow and shrink as per requirement without the need to maintain large capacity machines. Economies of Scale helps Netflix in this scenario.

The benefits are very clear, but it still took seven years for Netflix to complete the migration. Moving to the cloud is a lot of work and a lot of factors need to be considered. Netflix could easily move all of its existing systems to AWS but bringing existing systems also brings all the problems and limitations that were present. So, Netflix took the cloud-native approach, they rebuilt all of their technology and fundamentally changed the way they operate the whole company. Netflix migrated from a single application to thousands of micro-services.

Reasons for choosing AWS as the computing platform

Netflix has been fortunate to experience incredible growth in recent years, both in terms of customers and streaming devices. They started to hit that point where every layer of their software stack needs to be able to scale horizontally. With the shift to streaming, their software needs to be much more reliable, redundant, and fault tolerant.

After many fruitful years of growth on our previous three tier, database oriented architecture, it was time for them to go back to the drawing board. They could have chosen to build out new data centers, build their own redundancy and failover, data synchronisation systems, etc. Or, they could opt to write a check to someone else to do that instead.

So they move to cloud and letting Amazon to focus on data center infrastructure which allows their engineers to focus on building and improving their business.

Amazon calls its web services “undifferentiated heavy lifting”. The problems they are trying to solve are incredibly difficult ones, but they aren’t specific to business of Netflix. Every successful internet company has to figure out great storage solutions, hardware failover, networking infrastructure, etc.

Netflix want their engineers to focus as much of their time as possible on product innovation for the customer experience; that is what differentiates them from other competitors.

One year ago, their cloud computing expertise was limited to research and some prior experience at other companies. Today Netflix is running one of the highest volume cloud computing deployments in the world. Engineers who choose to work at Netflix are developing skills that will be increasingly relevant over the years to come as cloud computing becomes the dominant platform for growing internet companies.

How Netflix delivers contents to the users

It is through Open Connect. It is Netflix’s own Content Delivery Network (CDN) which it manages through Amazon. Videos that stream to a user are located in data center within the networks of Internet service providers, facilities where traffic is exchanged among most of the network operators. The traffic is distributed directly to Verizon, AT&T, Comcast, and similar network operators at such exchange points. When a user presses ‘play’ button, from these sites, videos get delivered to him.

Before a video gets delivered to him, operations like searching for videos and signing up by the user for the service are all handled in AWS cloud. Hence, the business logic, personalisation, search, and data processing which gives the streaming experience are all live in AWS. The technology had to maintain Netflix’s employees who were working in streaming business, and this was also housed in Amazon.

Netflix & AWS Lambda

Netflix gain new efficiencies using AWS Lambda. With lambda, they are able to replace inefficient procedural systems that pulls the infrastructure for updates in order to manage and control. So, that they can replace it with declarative rules based systems triggered when events happen that manage and adapt days of processing fabric to the needs and interest of their own.

How Netflix handle media files uploaded by content providers

Studios push media files frequently to Netflix data center. On receiving this file they chop it into five minutes chunks for parallel processing and then distribute across the systems and code. When the last piece is encoded, repackage it and then deploty it for the CDN.

With lambda, we can use rules triggered by the movement of those assets to launch and configure the necessary processing to encode them into 60 different parallel streams and then they use the rules and the events to aggregate and deploy after all the parts have been processed.

In the case of Backup, in the environment hundreds of processors save updated data in S3 continuously and with lambda, use the rules that trigger on the data updates that decides whats need to be backup and what needs to be copied to off-site storage and to check and valid that it arrived safely and to restart the copies and recheck and revalidate, if it didn’t raise alarms in case of failure.

How Netflix take security measures?

To handle the security, they have hundreds of different processes that start and stop instances all the time and lambda validate each new instance and is constructed and configured with the rules and situations. and to trigger shutdown the violations or notifications of unauthorized instances that appear in the infrastructure

Do you know why Netflix took seven years to shift to Amazon?

They rebuilt their entire software platform to leverage AWS cloud network to the maximum. ‘Chaos Monkey’ is a series of tools developed by Netflix to reduce damage in the case of disruptions. On the Christmas Eve of 2012, the company suffered a streaming failure and at the time it was on a single Amazon region. Since then, they have invested heavily in disaster recovery. Now, Netflix mainly operates in Oregon, Northern Virginia, and Dublin regions.

What if one of these regions goes down?

In this case, Netflix redirects the traffic to other available regions within a moment’s notice. The company has enough backups of all data which is stored in Amazon itself.

The distributed database, Cassandra is chosen to store customer data where every data element is replicated many times in production. The primary backups of all data are generated into S3 (Simple Storage Service). Any kind of operator errors, logical errors, software bugs, or other such corruptions can be dealt with by the S3 backups. ‘Armageddon Monkey’ is Netflix’s attempt to recover from failures of all its systems on AWS.

Netflix says about Amazon

“Elasticity of the cloud allows us to add thousands of virtual servers and petabytes of storage within minutes, making such an expansion possible.’ So, that is the power of Amazon Web Services propelling one of the most ambitious companies on earth, Netflix, into uncharted territory and runaway success!”

Today!

Netflix is the world’s leading internet television network, with more than 100 million members in more than 190 countries enjoying 125 million hours of TV shows and movies each day. Netflix uses AWS for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more — hundreds of functions that in total use more than 100,000 server instances on AWS.

One of the most well-known success stories of AWS is Netflix. The popular streaming service relies heavily on AWS to deliver content to its 200+ million subscribers worldwide. AWS provides Netflix with scalable, reliable, and cost-effective infrastructure that enables the company to deliver high-quality streaming services to its customers.

Netflix is one of the largest and most successful streaming services in the world, and it relies heavily on AWS to deliver its content to millions of users worldwide. Overall, Netflix’s success with AWS highlights the power and flexibility of cloud computing. By relying on AWS to handle its infrastructure, Netflix can focus on delivering high-quality content to its users and driving innovation in the streaming industry.

Here are some details about how Netflix uses AWS to achieve its success:

1. Content Delivery: Netflix uses AWS to deliver its content to users in the most efficient and cost-effective way possible. AWS provides Netflix with a global network of servers and content delivery networks (CDNs) that allow the company to stream high-quality video content to users in over 190 countries.
2. Scalability: One of the biggest advantages of using AWS is its scalability. Netflix can quickly and easily scale its infrastructure up or down as the demand for its services changes. This allows the company to handle huge spikes in traffic during peak hours, such as when new episodes of popular shows are released.
3. Cost-Effectiveness: AWS provides Netflix with a cost-effective infrastructure that allows the company to save on infrastructure costs while delivering a high-quality user experience. By using AWS, Netflix can avoid the high costs of building and maintaining its own data centers and instead pay only for the resources it actually uses.
4. Data Analytics: Netflix also uses AWS to analyze data and gain insights into user behavior. By using AWS’s analytics services, such as Amazon Kinesis and Amazon Redshift, Netflix can monitor user activity, identify trends, and make data-driven decisions to improve its services.
5. Innovation: Finally, by relying on AWS, Netflix can focus on innovating and developing new features rather than worrying about infrastructure. This has allowed the company to pioneer new technologies, such as its adaptive streaming technology, which optimizes video quality based on a user’s internet connection speed.