Automated Channel Testing - Plex Blog

Automated Channel Testing

, | 24 Apr 2011 | by 15 Comments

Along with the Plex desktop applications, there is a lot of infrastructure behind the scenes that we’ve been working on. Today I want to focus on Plex Channels, which bring Internet media content to your desktop and mobile Plex clients.

The Web is in a constant state of flux, with sites changing and being redesigned. Wherever possible, the Plex Channels use the site’s API to ensure stability in the face of this change, but at this point we have nearly 250 channels, and we decided it was time to add some automation to the quality assurance process. Whenever we have a server sitting with idle cycles, I’m deeply bothered, since I feel those cycles could be spent doing productive work. Didn’t someone once say that the devil makes work for idle cycles?

Fast-forward a few weeks and that lazy server is hard at work checking all of the channels for failures and putting the results up on a giant board with ~ wait for it ~ colored status indicators! That’s right, we took the colors that the US Dept of Homeland Security established for its terror alerts (doubtlessly arrived at after months of study and millions of dollars of expense), and shamelessly stole them for our leaderboard. According to the image below, if you plan on using Webcomics, expect a backscatter.

Plex Online  Plugin points 2

What you’re actually looking at in the list above are plug-ins that have been fixed. When the tester detects a problem, it uses advanced heuristics similar to those used by HAL when he tried to kill that annoying Dave character to assign a point value to the broken channel (How long as the plug-in been broken for? How badly broken is it? Is it a popular channel?). After that, our enterprising group of ninja developers swoops in and fixes it, the tester verifies the fix, and gives points to the developer who fixed it.

The points accumulate, and every month, the people with the top three scores have lavish amount of praise and modest cash prizes (enough to feed a healthy gadget obsession or impress a significant other) heaped upon them.

The first round just ended, and we’d like to take a few minutes to meet the winners. Amazingly, given the total number of points accumulated, there was a tie for first place (if that’s not the definition of bi-winning, I don’t know what is). Without further ado, let’s meet the first 1st place winner, Pierre (twitter: @pierredn), who had some beers with us during the San Jose Plex meet-up and who is an all-around super cool guy.

Pierre, can you tell us a bit about yourself?

I’m an electrical engineer by trade living in California. I’ve always loved computer programming and tinkering with computers, so Plex was a match made in heaven with its open plug-in architecture. Now the only thing I need is a bit of time to enjoy watching TV/Movies/Media ….

How long have you been using Plex?

I have been using Plex since May 2009 when I read an article about it, I was very intrigued and jumped right in. It was love at first sight and although my media library was pretty small at first due to the awkwardness of the media managers I used to use, it has now grown to a respectable size since Plex makes it so easy to manage all the different files you may have. After a few months of ramping up, I saw the lack of plugins in my native French language, started writing a few and offered to help. I haven’t stopped ever since. I now dream in Python.

What’s your favorite feature and the one thing you’d like to see changed/added in Plex?

favorite feature in Plex : Plug-ins! They helped me cut the cord on cable TV and save a ton a $$ every month. I now only watch what I want to watch and they help me stay connected to foreign media all the way to my bed with the great iOS app ….

one thing to change : more support/content for live events / sporting events – not entirely a Plex feature but that would the last step toward completely replacing traditional broadcasts and streaming absolutely everything.

Tied for first place is Sander, from the Netherlands, who has been Plexing for years and is one of the nicest and most helpful guys you will ever meet.

I’m a 30 year old guy, addicted to coffee, film festivals and TV shows such as Fringe, Twin Peaks, The X-Files and La Femme Nikita. I have a background in interaction design and usability testing, but my interests also include (new) web technologies and everything that’s got to do with (digital) video and film. When my Windows/MediaPortal PC died a little over two years ago, it didn’t take long for me to decide to switch to the Mac after I discovered Plex. Four months later I wrote my first plugin (for the Dutch Film Festival website). My favorite Plex feature is the automatic collection of metadata. Although I personally don’t like the media flags, I really love the background art and preview images that are shown for all my movies and shows. But the metadata settings is also a part of Plex that I see a lot of (non-English speaking) users struggle with and it could use some improvements.

In third place is Ryan McNally, who has also been with us for years; he’s written an iPhone app for Plex, and tons of plug-ins.

Ryan, can you tell us a little bit about yourself?

a/s/l: 29/male/Toronto, Canada

Likes: Curling, golf, Python and Objective-C. Really bad zombie movies. The black stuff. Comics that would put CCA censors into conniptions. Those forms of rock commonly described as ‘Hard’. Animation.

Dislikes: Allman braces, lower_case_variables_with_underscores, Flash Player for video streaming. Java apps on OS X. The perversion of the noble potato for wussy fruit drinks. Clown shoes.

Claims to Fame: Wrote the first mobile app to connect to the Plex Media Server. Some of his plug-ins have been downloaded over 9000 times. Charlie Sheen had to go into rehab after doing a hit of /him/.

How long have you been using Plex?

Since it was called osxbmc. Remember when it was called Plexsquared? And Sean was like “Drop the squared. It’s cleaner.” and we were all like “What the hell would you know Sean? Your p2p network is polluted and ‘Cry me a River’ is downright creepy.” And then we changed the name and went water-skiing with nude supermodels? At least that’s how I remember it.

What’s your favorite feature?

The media server is the best thing to happen to web media since H.264. One day people will wonder how we ever accessed web videos (or videos as they’ll call them) without it.

…and the one thing you’d like to see changed/added in Plex?

Well I’m not one to suggest changes to other people’s work but I’ll give it a go: I’d love to be able to write pms handlers for new file types (pdf, rar, cbz, epub to photosets/video streams) and webhosts (hotfile, nntp); create arbitrary metadata schemas (for comics, ebooks, magazines); and HID+streaming (GameCenter). I still have collections that aren’t managed by Plex. They laughed when I suggested ebooks in iTunes. Now it’s big business for Apple. I don’t want six apps to handle my entertainment. I want one, one good one. I think you know which one I’m talking about.

Many congratulations to our winners, and best of luck in the next round to all the others who participated! (By the way, the monthly competition is open to any developer who has at least three Channels/Agents accepted into the official Plex repo.)

FacebookTwitterGoogle+LinkedIn
Share this

15 Comments so far

  1. pv_bain April 24th, 2011 2:07 am

    Ha ha ha, great system ! I hope it will stay beers and not millions of bonus when you’ll be more than millionaires ;)

  2. Pete April 24th, 2011 6:51 am

    Is this channel checking page available to the general unwashed public? Would be handy to see if other people are having the same problems with certain plugins etc

  3. Itamar April 24th, 2011 6:58 am

    Can the board be seen on line or from plex?

  4. pooploser April 24th, 2011 8:17 am

    intersting and funny background infos! cool stuff! speaking of broken plugins… elan, could anyone of you guys have a look at the pitchfork plugin? harley and i wanted to one day, but i don’t think it we’ll ever be able to. would be great to have the world’s biggest independent music channel back in plex!

  5. Valconi April 24th, 2011 9:04 am

    What I really would like to see is a plugin for dutch radio. Shoutcast just doesn’t have all channels and the plugin looks terrible on the Ipad. Feed me causes crashes on Plex frontend all the time.

  6. Brent112 April 24th, 2011 10:43 am

    Congrats to the winners!! Now let’s just get flash/silverlight support on windows PMS
    so I can enjoy some of these channels. :-). Keep up the great work Dev’s.

  7. syncopation April 24th, 2011 4:03 pm

    Will this be public? Would be great. Transparancy is always good.

  8. Scott4142 April 25th, 2011 2:17 am

    Great article. I’ve recently cut over to strictly Internet television. I use Plex daily. I haven’t been able to get Hulu to stream most feeds? Have you tested the Hulu channel? Thanks and keep up the great work. Much appreciated.

  9. Smithcraft April 27th, 2011 10:08 pm

    Does this mean that Carpool will be working/get fixed?

    SC

  10. wmbell April 28th, 2011 3:27 pm

    Are the live results of the Automated Channel Tester available publicly? It seems as if this resource would be really nice, to know if it is just me or everyone having issues! =)

  11. Tobias May 3rd, 2011 5:51 pm

    What does this mean to us users? I don’t quite get it

  12. hmm May 7th, 2011 6:11 pm

    A few questions about if this is going public or not have nicely been ignored…

    The lack of decent updates from the devs around here is really getting annoying. Sure we all like big announcements every now and then but getting info out of devs is like getting blood out of a stone!!

    Transparency is a good thing, I’ve got no idea why everything has to be so secretive…

  13. elan May 7th, 2011 11:30 pm

    @hmm ~ We may have a public-facing side to this, yes, and that’s a good idea.

    Not sure what you mean by “decent updates”, we reply to questions over Twitter, in the forums, and in here. If I knew what questions you had, I might be able to answer them.

  14. wmbell July 10th, 2011 9:33 pm

    The Automated Channel Tester must not be working as it should, because there are several unresolved plugin issues out there.

  15. Sander July 13th, 2011 12:50 pm

    @wmbell: the tester only tests plugins, it doesn’t fix them. Devs are still needed to fix broken plugins.

Leave a reply