At the heart of all Standards Definitions Organizations (SDOs) and technical associations like the SVTA is testing. Building technology (such as open-source code) or developing a specific way to do something within a technology or set of technologies requires that ideas and assumptions be tested.
But the streaming video technology stack is fraught with complexity. While much of it is based on standard and industry-accepted ways of coding, authenticating, interoperability, and interfaces (APIs), there are still many different ways of building systems. What does this mean for the streaming video tech stack? It means there are a lot of different configurations for the various components…and lots of different components themselves. There isn’t just one vendor offering an encoder. It’s dozens along with open-source software. The combination of different vendors (and the many ways to configure their systems) along with a multitude of programming languages, cloud infrastructure providers, container approaches (for virtualization), makes it impossible to build industry-wide solutions, or standards, without testing.
The Challenges of Testing
While many of the streaming video tech stack components offer virtualized or containerized versions, it can still be difficult to properly test them. It can take days or even weeks to get a component properly deployed. And then, such as with encoders, they need to be properly configured which requires additional testing. But those configuration settings could be impacted by the kind of content being tested as well as downstream components. And that’s just the start of it. What about architectures that make use of physical equipment (like an ASIC-based encoder) or storage? This makes it incredible hard to test. And without being able to test assumptions made about different methods of doing things, the specifications and/or standards may only apply to a very small subset of use cases rather than the wider industry.
But these challenges are exacerbated by the testing environment itself. Setting up a physical lab can not only be costly but very time consuming. When the SVTA first set out to create a lab for working groups to test assumptions behind documents, we worked with a member company, Liberty Global, who had a physical lab they offered to allow us to use. Unfortunately, while it was a very generous offer, it ended up being more of a headache as it was difficult to coordinate resources (Liberty Global was CET and many of our participants were based in North America) and it was sometimes difficult to identify who was responsible for dealing with our requests.
Moving Testing To The Cloud (In A Standardized Way)
When we experienced the difficulties of trying to setup a testing environment in a physical lab, and recognizing how hard testing would be regardless if we needed to setup and configure elements each time, we set our sites on the cloud. Again, given that most vendors had virtualized or containerized versions of their solutions (or APIs for connecting to their cloud programmatically), this made perfect sense. But managing cloud environments can be just as difficult as managing on-premises labs, maybe even more so as billing, IAM, and other settings need to be configured to keep guardrails on the cloud resources. It’s not difficult to leave instances running or over-provision them resulting in thousands of dollars of wasted spend.
Our goal was to not only figure out a way to build a test environment in the cloud, but make it repeatable.
In our Q1 2025 Member Meeting in Tucson, Arizona, we announced the availability of our sandbox on the Google Cloud Platform. While we didn’t favor GCP over AWS or Azure or any of our other member clouds because of specific features, GCP was simply willing to work with us more closely, offering resources (when we had questions) and credits to offset spend. Dan Drew, of Fortium Partners (formerly of Didja.tv, an SVTA Grant Member), lead the development of the Sandbox, continuing much of the work he had been doing during his time in the Low Latency Working Group helping to setup testing resources in the Liberty Global lab.
Having experienced the headaches of setting up a single workflow in a physical lab, Dan recommended and implemented and entirely Terraform-based solution in Github. Using a Terraform template for vendor technologies, member companies that wanted to participate in the Sandbox by including their product offerings or services could do so easily by simply adhering to the template and providing a GCP-compatible version of their product.
Not Just Components, But Content Too
What would a testing environment for a streaming video tech stack be without content? That’s why we have worked with other member companies, such as Sky (part of Comcast), to make available live feeds that working groups can use in their tests. In addition, we have produced a large library of VOD content in multiple bitrates, qualities, and even some that is deliberately malformed. Having content available ensures that testing is done as close to real-world as possible. This is also why some of the available components, like TAG Video Systems and Datazoom, are service based as many streaming operators incorporate third-party service providers and bump-in-the-wire components.
Commercial, Open-Source, and Standards-Based
In addition to the commercial technologies that are available as Terraform recipes, we also include open-source components, such as dash.js, video,js, and hls.js, as well as standards, like CMCD, that working groups can opt to include in their end-to-end workflows. This, again, serves not only to provide a means to test non-commercial software with commercial tools, but also to mimic for real-world scenarios. Many streaming operators utilize open-source components within their streaming video tech stack, such as FFMPEG and Gpac. And, they are utilizing technologies based on industry standards. For example, CTA Wave has produced two standards, Common Media Client Data (CMCD) and Common Access Token (CAT) that many streaming operators (the former) or CDNs (the later) are looking to implement. So if a working group wants to include a player, they will most like also want CMCD implemented within the player. And they may also want to test out the delivery of content with tokens that adhere to CAT standards. Of course, the SVTA has its own open-source projects that will be included, such as dash.js. We will even make an entire Open Caching CDN implementation available as a Terraform recipe in the event that testing involves delivery through a uCDN (i.e., an origin shield).
We Are Only At Phase 1
While we will continue to iterate and improve upon the Terraform-based solution that is in place, we also want to make the Sandbox even more accessible to our Working Groups and even our back office staff. The next phase of the Sandbox will look to implement a web-based interface, abstracting the CLI currently used for implementation. This UX will allow users to simply check boxes for the components they want, specify time periods, define project names, etc. All of the data elements captured from the interface will populate the .yaml configuration file and kickoff the execution script.
Of course, one of the biggest administration challenges for managing cloud-based resources in GCP, AWS, or somewhere else, is budgeting, access, and reporting. We don’t just want any WG going into the Github repo and spinning up a workflow without back office involvement. So the third phase will be about creating a resource request feature. WG chairs or project leads will make a request through that same web-based interface. The request will be passed to administrative personnel who can review and approve the project. Once approved, the members can return to the UI and specify the components they want in their workflow and other project details. The difference is that once the resources have spun up, administrators will have a new dashboard where they can see which projects are active as well as their budget and current spend.
A Better Way To Test, A Better Way To Build
We believe that this is a real first-of-its-kind amongst technical associations but our objective wasn’t to be bleeding edge or set a trend, it was to provide our working groups a much easier means to test the assumptions they were making, the best practices they were recommending, the optimizations they were suggesting. By leaning into testing, and making it easy to test, we are ensuring that the output from SVTA working groups is believable and measurable. This will ensure that companies wishing to use our specs or best practices to shape their own streaming workflows can do so with confidence that what’s being specified or recommended has been tested.