Analytics Segmentation: Ad Impressions

Last August, our data exploration tool Segmentation launched and has helped Advanced Analytics users take control of their data and dig deeper into video performance trends.

This month, we rolled out an important new metric: Ad Impressions!  

 

 

Users can now

Agile Incident Response for Big Data Systems

How We Increased Uptime and Decreased Engineering Stress

Working in the big data world can be a little chaotic. For us that means a stable flow of 20k events per second can spike to 50k over the course of an hour. It takes only seconds to look through our logs and find a perfect example:   chart   Things move fast and we need to be able to adapt and respond immediately when our systems are under stress. When a system goes “red” most people’s first instinct is to call the engineer who built it. Over time this can create silos of knowledge and an uneven load on the engineers. Another side-effect of this engineer-first thinking is that the priority is focused on fixing the problem, missing the importance of communication.

JW Recommendations – Powerful Content Discovery

It’s been a year since JW Recommendations launched and since then the product has been delivering millions of extra plays a day to publishers. This week’s release of Video Search Playlists and an enhanced web interface greatly improves our flexible & powerful content discovery solution.

Big Gains in 2016

Recommendations are a crucial part of many video publishers’ offerings and that has put a premium on simple and integrated solutions that can be delivered cost effectively at scale. JW Recommendations integrates deeply with each part of JW Player to provide a powerful, real-time and automated content discovery solution built on our vast network of viewer and content data streams.   recs-pic-1  

My Internship at JW Player

This summer I was very lucky to join JW Player as an engineering intern on the Data Team. It has been a fantastic experience. Aside from sailing on the Hudson River, enjoying Ping Pong games, and cycling on the Governors Island, I learned about their state-of-the-art data pipeline, followed Agile practices, and worked with an amazing group of people. I was part of the Discovery squad of the Data Team, worked on evaluating recommendation systems, and was responsible for developing an evaluation tool for our data-driven recommendations. With data-driven recommendations, we want to show our users relevant videos to increase video plays and user engagement. The question is how to evaluate if the recommended content is relevant, and which metrics to use as the measure. Generally there are three methods for evaluating recommendation systems: offline experiments, online trials, and user studies. In this project, we are using the user study approach, by directly asking the opinions of the viewers whether the recommended video is relevant or not.  

Architecture of the evaluation tool danmeng1

Introducing our newest JW Player Analytics dashboard feature: “Segmentation”!

segmentation JW Player’s Analytics dashboard is focused on helping online video publishers track key, top-level video metrics in real-time.  Our goal is to empower users to make quick decisions to drive audience growth and viewer engagement. In order to empower our publishers, we needed a way to give them more control over analysis.  Instead of only pre-defined dashboard widgets, users can define their own queries against historical data and see instant results.  Our easy-to-use interface gives non-technical users the answer to key questions about their audience and content.

An Easier Way to Run Spark Jobs on AWS EMR

spark-logo-trademark At JW Player, we use Spark to explore new data features and run reports that help drive product decisions and improve algorithms. But doing data analysis at the terabyte level is time consuming, especially when having to manually set up AWS Elastic Mapreduce (EMR) clusters. Our code often depends on custom libraries or Spark settings that require bootstrapping. Moreover, iterating on changes is cumbersome and adds extra steps to our workflow.

JW Platform’s Data-Driven Recommendations & Feeds

Franklin Dement, Product Manager for JW Platform’s Data-Driven Recommendations, explains new features as well as best practices for implementation.

Why use Data-Driven Recommendations?

Launched in November 2015 to a select group of publishers, JW Platform Data-Driven Recommendations:
  • Increase content plays and monetization opportunities where ads are present.
  • Use existing content connections and viewer activity to simplify your publishing workflow and increase revenue.
  • Provide your viewers with automatically-curated video suggestions that increase audience engagement and are continually updating based on interaction data and content traits.

JW Player In-Player Overlay — Available in Nov 2015 Release

player-img

Grow Your Audience With Our New JW Player Real-Time Analytics

In the new JW Player Analytics Dashboard, we’ve changed the way we analyze data in an effort to empower you with the information you need to grow.

FEATURE SPOTLIGHT:  Your New ‘Right Now’ view

In the fast-paced (and short-attention-span!) publishing world, we know that you are always searching for a quick and easy way to check what’s going on “right now” with your content campaigns. How many times have you realized a video was trending after the moment passed and your audience moved on? Static dashboard tools with stale data can’t solve for this use-case. REALtime new

Hive with Tez on EMR

Screen Shot 2015-12-11 at 3.51.19 PM Over half a billion videos are watched on JW Player video player every day resulting in about 7 billion events a day which generates approximately 1.5 to 2 terabytes of compressed data every day. We, in the data team here at JW Player, have built various batch and real time pipelines to crunch this data in order to provide analytics to our customers. For more details about our infrastructure, you can look at JW at Scale and Fast String Matching. In this post, I am going to discuss how we got Hive with Tez running in our batch processing pipelines.

Fast String Matching for Analytics Quality at JW Player

At JW Player our analytics pipeline currently receives 4M pings per minute at peak times, providing the basis for insights to the publishers on our dashboards. Recently we have moved some of our offline classification processes to the beginning of our real-time and batch pipelines, which required us to optimize our string matching implementations. We employed finite state automata, rolling hash functions, and bloom filters to achieve 6x and 3x speedup in NSFW classification and ad classification respectively. In this article, we will discuss these classification problems, then the algorithms, implementations, as well as evaluation of our solutions.Classification_omni