Frequently Asked Questions

What is Puffer?

Puffer is a Stanford University research study about using machine learning to improve video-streaming algorithms: the kind of algorithms used by services such as YouTube, Netflix, and Twitch. We are trying to figure out how to teach a computer to design new algorithms that reduce glitches and stalls in streaming video (especially over wireless networks and those with limited capacity, such as in rural areas), improve picture quality, and predict how the capacity of an Internet connection will change over time.

Sounds interesting. What can the public do to help?

Watch TV on this website. The idea of this study is streaming TV channels to study participants over the Internet, and the Puffer website will automatically experiment with different algorithms that control the timing and quality of video sent to them, and will monitor how well the resulting computer-designed algorithms work. The more diverse the Internet connections that the study participants use, the better the system will be able to learn, and the more robust the resulting computer-generated algorithms.

So you're doing a research project to improve video streaming technology, and my role is to watch television on your website?

Yes. Well, technically you don't even have to watch. We just need people to stream video over different kinds of Internet connections, so that the computer has some live traffic to learn from and experiment on.

How can I join the study and watch TV on your website?

Visit the Sign up page to join. You must be within the United States to use Puffer.

Do I have to pay?

No, there is no charge to participate. Puffer is an academic project at Stanford University and is entirely non-profit.

What channels can I watch?

Puffer re-transmits free over-the-air broadcast television signals received by an antenna located on the campus of Stanford University. At the moment, we are re-transmitting local affiliates or owned stations of CBS (KPIX 5), NBC (KNTV 11), ABC (KGO 7), Fox (KTVU 2), PBS (KQED 9), CW (KBCW 44).

Do you make any changes to the local TV channels?

No, Puffer retransmits the local stations we can receive without any modification.

Can you please add HBO/CNN/ESPN?

No, we can only re-transmit free over-the-air broadcast television signals.

Why doesn't Puffer work on my iPhone or iPad?

Puffer uses the Web Media Source Extensions (MSE) to stream video. This standard is not supported by all browsers, and in particular, it is not supported by Safari on iOS (and we understand other browser engines are not allowed on iOS, which means it is not possible to watch Puffer on an iPhone or iPad). Puffer does work in Chrome, Firefox, Microsoft Edge, and Opera (including on Android phones and tablets). If you are having trouble, please contact the research team by visiting the puffer-stanford Google Group.

Why doesn't Puffer work on Safari on my desktop Mac?

Puffer uses the Opus audio standard, which Safari supports only for real-time video (WebRTC) but not for video streaming (in a WebM container).

So... what browsers does Puffer work on again?

Chrome, Firefox, Opera, and Microsoft Edge, on Mac/Windows/Linux computers and on Android phones and tablets.

Why do I keep seeing connection timeout errors?

The short answer: If Puffer is not in an active tab being directly viewed, we can't guarantee how your browser/device will behave. If you play Puffer in the background and see this error upon returning, please just refresh the page.

The long answer: Puffer requires clients to frequently communicate with the server to avoid timeouts. Further, as a live streaming service, Puffer does not support pausing. However, many mobile devices implement mechanisms to force video pausing regardless of application support. Such pausing happens in response to switching from Puffer to other applications, or pausing video via the notification center. When such pausing occurs, server timeouts may occur. Additionally, poor network conditions can cause timeouts, as can leaving Puffer open in a background tab for browsers which restrict the resource consumption of background tabs.

Can I help your research project by playing Puffer in a background tab?

Different browsers have different policies for how they throttle the performance of background tabs. Even if your browser is capable of playing video in the background, video performance may be reduced. As a result, playing Puffer in the background does not provide us with useful data.

When you say Puffer will learn better video-streaming “algorithms,” which algorithms do you mean?

Puffer is focused on three types of algorithms: “congestion-control” algorithms that decide when to send each piece of data, also known as a packet, “throughput forecasters,” which predict how long it will take to send a certain amount of data over an Internet connection in the near future, and “adaptive-bitrate” (ABR) algorithms that decide what quality of video to send, in order to try to give the user the best picture quality that won't lead to a stall or rebuffer. All three kinds of algorithms have a long history of work by academics and practitioners.

Why do you need a website with people streaming video from you to do this research?

The core issue here is that developing robust systems takes data and experience. Big companies (e.g., YouTube, Netflix, Twitch) have lots of users, which means lots of data: they could, at least in theory, do experiments where some users get one variant of an algorithm for a day, and some get a different variant, and at the end of the day there is tons of data to decide which variant is better to the bottom-line metrics (generally in terms of things like stalls and startup speed, picture quality and the variation in quality over time, and cost or wasted bandwidth).

The challenge for the big companies is that they are generally pretty conservative about changing their low-level systems code, like congestion-control algorithms and throughput prediction algorithms. Letting a machine learn these algorithms online, by running a new experiment every hour or so, seems to be a nonstarter for these services—there's too much at risk.

Academics, on the other hand, historically have had to develop network algorithms in simulation or in small real-world testbeds, sometimes known as “offline” learning. Unfortunately, we have learned through experience that algorithms learned in simulation or in small-scale experiments sometimes aren't able to replicate their performance in the real world, or even in subsequent small-scale experiments. We're not sure if it's possible to learn a network algorithm in simulation and have it perform robustly on real traffic over the real-world Internet. (Nobody has figured out how to do this yet, which probably indicates a failure of our simulations, but it's not obvious where.) To teach a machine to develop robust algorithms automatically, academics need a real population of Internet connections we can experiment on.

Puffer is attempting to answer the question of whether rapid “online” learning—automatically generating and testing algorithms on real users in situ—can successfully produce robust systems. Hence this website: a medium-sized video streaming website, open to the public, run for academic purposes, that learns low-level systems algorithms as a function of who is using it.

Why are you streaming over-the-air television?

For the research project to work, we need some source of video that never ends and is sufficiently interesting to attract a few hundred people to watch. Broadcast TV is the most interesting thing that we have access to and are permitted to retransmit.

Can you share more technical details about how Puffer works?

Sure, please check out our research paper and the source code of Puffer.

Who is responsible for Puffer?

The Puffer project is led by Francis Yan, a doctoral student in computer science at Stanford University, along with Hudson Ayers and Sadjad Fouladi (also Stanford) and Chenzhi Zhu (Tsinghua University). The project is advised by professors Keith Winstein and Philip Levis.

The project has been funded in part by the NSF and DARPA, along with support from Google, Huawei, VMware, Dropbox, Facebook, and the Stanford Platform Lab.

Does the research group behind Puffer have any past work in this area?

We have recently published work on neural networks and a standardized training ground for congestion control and on better systems for real-time Internet video and on high-speed video encoding using thousands of tiny threads. In earlier days, the advisor was responsible for work in computer-generated congestion control and in forecasting the speed of an Internet connection for video and integrating application coding and congestion control.

What information do you collect about me?

As you stream video, the Puffer website will automatically collect information from your Web browser, including which video channel was sent to you, the picture quality of that video compared with the original version received by our antenna, the timing and duration of stalls in the playback, the timing of the delivery or loss of individual Internet Protocol datagrams or messages, and the accuracy of predictions the Website makes about the capacity of the Internet connection between you and the Website.

Is there a survey to fill out?

No, you will not receive a survey or be required to answer any questions as part of the study.

Does this study count as “human-subjects research”?

No, the Stanford University Institutional Review Board has determined that this project does not meet the definition of human subjects research as defined in federal regulations 45 C.F.R. § 46.102 or 21 C.F.R. § 50.3.

I lost my password. Can I reset it?

No, Puffer doesn't collect any private information about research subjects, not even their email addresses. As a result, it is not possible to reset a lost password. If the Puffer website is still accepting new research subjects, you can try to create a new account.

What are the limitations on study participants?

You must be within the United States to use Puffer. The study is limited to 500 participants simultaneously, meaning that at most 500 people are allowed to watch Puffer at a time. We expect each participant to stream video from the Website for several hours each week, and we reserve the right to terminate your participation if you do not meet this expectation. You can't copy, distribute, rebroadcast, etc., anything you receive from us. Please read the formal Terms of Participation before signing up.

Video streaming already works—isn't this a solved problem?

It does work for a lot of people! Google has published that 93% of YouTube sessions never stall to rebuffer. Whether this means video streaming is a solved problem somewhat depends on your perspective. If you are on a wireless connection or in a rural area, your results are probably worse. And video streaming necessarily involves a tradeoff between picture quality and the risk of stall. If you care about the picture quality (and not just whether the video plays without stalling to rebuffer), your demands probably increase with time, especially as video grows to greater resolutions and sizes, so even today's wired networks will prove inadequate in the future.

Puffer is unique from previous academic studies on video streaming in that it uses online learning to generate adaptive bit rate (ABR) and congestion control algorithms. In essence, this means that Puffer regularly learns from past performance to construct future algorithms which work better. Ultimately, we hope that this approach will produce substantial benefits over prior work, but only time will tell. Beyond this fundamental difference, Puffer also approaches the video-streaming problem differently in a number of ways. These include:

  • Puffer's use of structural similarity (SSIM) instead of bitrate as the input to quality of experience measurements, ensuring performance measurements are more directly linked to user experience.
  • Puffer's use of a denser bitrate ladder than most existing systems, allowing for more fine-grained control of video quality received by users.
  • Use of websockets instead of “DASH” HTTP request/response pairs, allowing for continuous streaming video that is not synchronous with client requests.
  • Using congestion control with a tunable pacing rate and a direct access to a throughput estimate, instead of having all testing occur on top of TCP.
  • Verbose communication between the congestion control layer and the application layer, so that application layer decision about video quality can be informed by the bandwidth available in the congestion control layer.
  • The use of nontraditional signals to train our transmission-time predictor, such as ISP, connectivity type, etc.
  • Using a direct Transmission Time Predictor (which predicts the time required to send a chunk of a certain coded length) instead of simply extrapolating a unitary throughput estimate.
  • Retraining the Transmission Time Predictor day-by-day as a function of real data.
  • Automatically generating variants of the ABR scheme, and evolving them day-by-day as a function of real data.
  • Using joint control across different users vs. treating each flow separately.
How can I ask a question not answered here?

Please contact the research team by visiting the puffer-stanford Google Group, or send an email to our mailing list "puffer [at]".