Puffer is a Stanford University research study about using machine learning to improve video-streaming algorithms: the kind of algorithms used by services such as YouTube, Netflix, and Twitch. We are trying to figure out how to teach a computer to design new algorithms that reduce glitches and stalls in streaming video (especially over wireless networks and those with limited capacity, such as in rural areas), improve picture quality, and predict how the capacity of an Internet connection will change over time.
Watch TV on this website. The idea of this study is streaming TV channels to study participants over the Internet, and the Puffer website will automatically experiment with different algorithms that control the timing and quality of video sent to them, and will monitor how well the resulting computer-designed algorithms work. The more diverse the Internet connections that the study participants use, the better the system will be able to learn, and the more robust the resulting computer-generated algorithms.
Yes. Well, technically you don't even have to watch. We just need people to stream video over different kinds of Internet connections, so that the computer has some live traffic to learn from and experiment on.
Visit the Sign up page to join. You must be within the United States to use Puffer.
No, there is no charge to participate. Puffer is an academic project at Stanford University and is entirely non-profit.
Puffer re-transmits free over-the-air broadcast television signals received by an antenna located on the campus of Stanford University. At the moment, we are re-transmitting local affiliates or owned stations of CBS (KPIX 5), NBC (KNTV 11), ABC (KGO 7), Fox (KTVU 2), PBS (KQED 9), CW (KBCW 44).
No, Puffer retransmits the local stations we can receive without any modification.
No, we can only re-transmit free over-the-air broadcast television signals.
Puffer uses the Web Media Source Extensions (MSE) to stream video. This standard is not supported by all browsers, and in particular, it is not supported by Safari on iOS (and we understand other browser engines are not allowed on iOS, which means it is not possible to watch Puffer on an iPhone or iPad). Puffer does work in Chrome, Firefox, Microsoft Edge, and Opera (including on Android phones and tablets). If you are having trouble, please contact the research team by visiting the puffer-stanford Google Group.
Puffer uses the Opus audio standard, which Safari supports only for real-time video (WebRTC) but not for video streaming (in a WebM container).
Chrome, Firefox, Opera, and Microsoft Edge, on Mac/Windows/Linux computers and on Android phones and tablets.
The short answer: If Puffer is not in an active tab being directly viewed, we can't guarantee how your browser/device will behave. If you play Puffer in the background and see this error upon returning, please just refresh the page.
The long answer: Puffer requires clients to frequently communicate with the server to avoid timeouts. Further, as a live streaming service, Puffer does not support pausing. However, many mobile devices implement mechanisms to force video pausing regardless of application support. Such pausing happens in response to switching from Puffer to other applications, or pausing video via the notification center. When such pausing occurs, server timeouts may occur. Additionally, poor network conditions can cause timeouts, as can leaving Puffer open in a background tab for browsers which restrict the resource consumption of background tabs.
Different browsers have different policies for how they throttle the performance of background tabs. Even if your browser is capable of playing video in the background, video performance may be reduced. As a result, playing Puffer in the background does not provide us with useful data.
Puffer is focused on three types of algorithms: “congestion-control” algorithms that decide when to send each piece of data, also known as a packet, “throughput forecasters,” which predict how long it will take to send a certain amount of data over an Internet connection in the near future, and “adaptive-bitrate” (ABR) algorithms that decide what quality of video to send, in order to try to give the user the best picture quality that won't lead to a stall or rebuffer. All three kinds of algorithms have a long history of work by academics and practitioners.
The core issue here is that developing robust systems takes data and experience. Big companies (e.g., YouTube, Netflix, Twitch) have lots of users, which means lots of data: they could, at least in theory, do experiments where some users get one variant of an algorithm for a day, and some get a different variant, and at the end of the day there is tons of data to decide which variant is better to the bottom-line metrics (generally in terms of things like stalls and startup speed, picture quality and the variation in quality over time, and cost or wasted bandwidth).
The challenge for the big companies is that they are generally pretty conservative about changing their low-level systems code, like congestion-control algorithms and throughput prediction algorithms. Letting a machine learn these algorithms online, by running a new experiment every hour or so, seems to be a nonstarter for these services—there's too much at risk.
Academics, on the other hand, historically have had to develop network algorithms in simulation or in small real-world testbeds, sometimes known as “offline” learning. Unfortunately, we have learned through experience that algorithms learned in simulation or in small-scale experiments sometimes aren't able to replicate their performance in the real world, or even in subsequent small-scale experiments. We're not sure if it's possible to learn a network algorithm in simulation and have it perform robustly on real traffic over the real-world Internet. (Nobody has figured out how to do this yet, which probably indicates a failure of our simulations, but it's not obvious where.) To teach a machine to develop robust algorithms automatically, academics need a real population of Internet connections we can experiment on.
Puffer is attempting to answer the question of whether rapid “online” learning—automatically generating and testing algorithms on real users in situ—can successfully produce robust systems. Hence this website: a medium-sized video streaming website, open to the public, run for academic purposes, that learns low-level systems algorithms as a function of who is using it.
For the research project to work, we need some source of video that never ends and is sufficiently interesting to attract a few hundred people to watch. Broadcast TV is the most interesting thing that we have access to and are permitted to retransmit.
Sure, please check out our research paper and the source code of Puffer.
The Puffer project is led by Francis Yan, a doctoral student in computer science at Stanford University, along with Hudson Ayers and Sadjad Fouladi (also Stanford) and Chenzhi Zhu (Tsinghua University). The project is advised by professors Keith Winstein and Philip Levis.
The project has been funded in part by the NSF and DARPA, along with support from Google, Huawei, VMware, Dropbox, Facebook, and the Stanford Platform Lab.
We have recently published work on neural networks and a standardized training ground for congestion control and on better systems for real-time Internet video and on high-speed video encoding using thousands of tiny threads. In earlier days, the advisor was responsible for work in computer-generated congestion control and in forecasting the speed of an Internet connection for video and integrating application coding and congestion control.
As you stream video, the Puffer website will automatically collect information from your Web browser, including which video channel was sent to you, the picture quality of that video compared with the original version received by our antenna, the timing and duration of stalls in the playback, the timing of the delivery or loss of individual Internet Protocol datagrams or messages, and the accuracy of predictions the Website makes about the capacity of the Internet connection between you and the Website.
No, you will not receive a survey or be required to answer any questions as part of the study.
No, the Stanford University Institutional Review Board has determined that this project does not meet the definition of human subjects research as defined in federal regulations 45 C.F.R. § 46.102 or 21 C.F.R. § 50.3.
No, Puffer doesn't collect any private information about research subjects, not even their email addresses. As a result, it is not possible to reset a lost password. If the Puffer website is still accepting new research subjects, you can try to create a new account.
You must be within the United States to use Puffer. The study is limited to 500 participants simultaneously, meaning that at most 500 people are allowed to watch Puffer at a time. We expect each participant to stream video from the Website for several hours each week, and we reserve the right to terminate your participation if you do not meet this expectation. You can't copy, distribute, rebroadcast, etc., anything you receive from us. Please read the formal Terms of Participation before signing up.
It does work for a lot of people! Google has published that 93% of YouTube sessions never stall to rebuffer. Whether this means video streaming is a solved problem somewhat depends on your perspective. If you are on a wireless connection or in a rural area, your results are probably worse. And video streaming necessarily involves a tradeoff between picture quality and the risk of stall. If you care about the picture quality (and not just whether the video plays without stalling to rebuffer), your demands probably increase with time, especially as video grows to greater resolutions and sizes, so even today's wired networks will prove inadequate in the future.
Puffer is unique from previous academic studies on video streaming in that it uses online learning to generate adaptive bit rate (ABR) and congestion control algorithms. In essence, this means that Puffer regularly learns from past performance to construct future algorithms which work better. Ultimately, we hope that this approach will produce substantial benefits over prior work, but only time will tell. Beyond this fundamental difference, Puffer also approaches the video-streaming problem differently in a number of ways. These include:
Please contact the research team by visiting the puffer-stanford Google Group, or send an email to our mailing list "puffer [at] cs.stanford.edu".