AOMedia Member Spotlight with Twitch's Principal Research Engineer, Video, Yueshi Shen

Please introduce your company and yourself briefly.

Twitch is a user-generated-content (UGC) live streaming platform with more than 100 thousand concurrent channels and 4 million concurrent viewers at its peak. Twitch enables a huge number of individual content creators to share their live experience (gaming, music playing, traveling, etc.) with the community. Typically, broadcasting on Twitch involves interaction between a broadcaster and his/her audience through chat messages. Such a "lean-forward" experience differentiates Twitch from the traditional linear TV broadcasters.

I am in charge of Twitch's core video technologies. I initiated and built a number of Twitch's core video capabilities, e.g., a cost-effective live-video transcoding farm supporting over 100,000 concurrent channels, a live ABR playback algorithm designed for highly interactive content, and HLS-based low-latency (<5s) live HTTP streaming. I am also the inventor of AV1's Switch Frame and have 7 patents in areas of codec technology, low-latency live broadcast, and ABR streaming.

What first brought you to AOMedia, and why did you decide to become a member?

I started my career in codec back in 2005. I have worked on many video formats since then including MPEG-2, H.264, HEVC, VP8, and VP9. It has been quite frustrating for many codec engineers including me to observe the royalty scheme chaos of HEVC and the market split of HEVC vs. VP9. Since the AV1 project was initially announced in 2015, I participated in the AV1 standardization process representing Twitch/Amazon. I became super excited about the idea of the technical community creating a new codec format that is royalty-free, offers the best user experience, and is specially designed for streaming video on the Internet. With a joint effort being made by technical and business leaders from multiple Amazon companies, Twitch and Amazon are trying our best to influence the ecosystem towards having a new codec format, which is supported across all client platforms including desktop, mobile devices, smart TVs, set-top boxes, game consoles, etc.

What are you currently working on with regard to AV1? How will end-users benefit from your work? Businesses? Markets?

Twitch believes that the most critical factor for AV1 to gain wide adoption is its ability to enable a new user experience for the OTT live streaming use case. We have established a cross-industry R&D collaboration to prototype a solution – based on the AV1 Switch Frame feature – that combines ultra-low (<1.5s) glass-to-glass latency with a higher QoS (i.e., lower buffering rate), that will provide an unprecedented interactive user experience for Esports and social media content. In addition, we are working with other partners to demonstrate 1440p120, which is a new viewing experience that our gamer community cares more about than 4K30 or 4K60 for the content they enjoy and the display devices they typically use to consume video.

What AOMedia efforts are you most excited about?

I am glad to see AOMedia bring together companies in different industries (SoC, consumer devices, cloud solution providers, content platforms, etc.) to create a healthy AV1 ecosystem. A new codec format is long overdue and unfortunately, H.264, a 17-year-old standard, is still the only option for most businesses wanting to stream video. On the other hand, a new codec format is commercially feasible only if every player in the market can realize its own business benefit from the technology. I am excited about the future of AV1 and believe it has the best chance to become the next most popular video format in the foreseeable future.