Continuing the Conversation - Plex Blog

Continuing the Conversation

| 28 Aug 2010 | by 48 Comments

So I’d like to hear a bit more about these scanners. That wasn’t a question, but sure. They are little snippets of Python code that are invoked by the Plex Media Server when scanning a library section. They are passed a list of files, and a list of subdirectories, and return a list of media objects. They look at the list of files, convert ones they recognize into media objects, and are also allowed to remove any subdirectories they don’t want to recurse into. Also, they can be “stacked” or “pipelined”, so the Movie scanner first calls the “Video Files” scanner to eliminate all the non-video files, and ends by calling the “Stack” scanner which stacks those pesky CD1/CD2 files, for example. Modular, extensible, and flexible.

That meant absolutely nothing to me, can you tell me something cool from the end-user perspective? Sure. As written, the TV scanner and the Movie scanner both take advantage of each other so that I can actually add the same “Incoming” directory to both my Movie and TV library sections, and they each pick out just the media that they’re supposed to. It’s not perfect, but it works pretty well.

Wait, but what happens when I move my media from my Incoming directory into its final resting place? Alexandria uses “turbo hashes” to keep track of files, so it’ll detect the fact that you moved the file, and you really won’t notice, it’ll just work.

OK, riddle me this! What if I have media on an external drive, unplug it, and then scan. Surely all hell will break loose and the Media Server will crash, choke, or delete my content, right? Right?! You mistake us for amateurs, kid. We just note that the location is offline and don’t whack anything.

So I have to keep updating my library manually? Only if you want to. You can update each section manually, hit a URL (if you’re the scripting type), have the library update periodically (every hour, say), or even enable “fs_events”, which auto-magically scans your library when something changes. Doesn’t work perfectly yet, but it’s pretty magical when it does.

I made a movie section, watched some stuff, rated some stuff, and then I accidentally deleted my section, or maybe I didn’t have it configured right, or maybe I just felt like recreating and scanning from scratch. Somehow, all my view status, ratings, and even position in the file wasn’t lost. Is this magic? Yes.

But I even renamed some movies in the process. And moved them elsewhere. How the @#$%@ does that work? Magic.

I’m really not sold on this centralized server thing. A bunch of my movies are located on my “server” machine, but I also have a bunch on the machine where I have Plex. Isn’t that horrible inefficient? As long as the paths are the same on both machines, Plex will simply use the local path for media when it’s available. We call this “optimal media routing”. (It’s actually the only way that VIDEO_TS movies work now, but that’s an aside.)

But I’m a UNIX hacker and I have Chinese food in my beard. Command lines FTW! The Plex Media Scanner can actually be run from the command line and do a variety of things like scan, refresh, list sections, add new sections, etc.

What’s your favorite metadata agent? It would have to be the IMDB agent written by the brilliant and handsome Scott. It’s not perfect, but it’s almost frighteningly good. An anecdote from a recent tester explains it better than I ever possibly could: “I had a big pile of old Danish movies, some which contain certain 18+ content….Plex/Eight recognized them all as this one weird English documentary program with an innocent icon. Plex/Nine recognizes and identifies it correctly – exposing rather explicitly that I had this old and forgotten folder of movies….AMAZING“.

But when the site changes, it’ll break and I’ll complain. The new agents auto-update just like the plug-ins, so as soon as we fix it, everyone will instantly benefit.

Well, this all sounds too good to be true. Can you explain some of the rough edges? Sure! Like I said, VIDEO_TS movies need to be accessible via the local filesystem, they don’t work over HTTP yet. VIDEO_TS files don’t scan properly for TV shows yet (due to my personal hatred for the format?). Let’s see, what else? You can’t specify DVD order vs aired order at the moment. The Music Scanner is pretty slow. I’m sure there’s more, but you’ll tell us about it soon enough.

What about date-based shows? Well, I’m sure it’s not perfect, but this folder of junk scanned in perfectly. And if you haven’t seen the 2005-01-25 episode, I highly recommend it, Rob Cordry at his absolute best.

The Daily Show.jpg

So I added some sections, and now I’m totally mesmerized watching the Media Manager. Can you explain what it’s doing? Yes, the Core Animation viewer is quite fun to watch. Obviously, the starting point is the gray Plex icon. When it starts looking for a match, it gets an “atom” icon overlaid. If it finds a match, it goes dark while the metadata loads, which can actually take a while. After it’s loaded, it flips over with the poster. Also, since the media analysis happens in parallel, sometimes the gray icon turns into a thumb before the metadata is loaded as well, just depends on the order in which things happen. This image below shows the three stages.

Plex Media Manager.png

It takes forever! I’m impatient! The posters don’t show up in Plex! WTF? BBQ! There’s an easy solution for this one. Get a nice cold microbrew, put on some good music, and watch the Media Manager, while sitting in Lotus position. Watch it. Become one with it.

It doesn’t do movie collections yet! I can’t edit the metadata manually! I’m pissed off. I want my money back. Remember what I said about this being the ground floor level of the building? Give us a chance to add a few more floors, you won’t be disappointed.

I’ll leave you with Barkley, enjoying a lilikoi. Namaste.

Aperture-1.jpg

FacebookTwitterGoogle+LinkedIn
Share this

48 Comments so far

  1. Roopesh August 28th, 2010 10:08 pm

    I didn’t know there’s a money back guarantee! :)

    Hey I’m really looking forward to P9 and as always appreciate the work you and the team put into Plex.

  2. Rylin August 28th, 2010 10:08 pm

    Huge thumbs up!

    Does Plex.9 include a yoga mat and Advil, for those times one experiments with the Lotus position and micro brews?

  3. Kane610 August 28th, 2010 10:10 pm

    I’m OD:ing here with this sudden occurence of information.

  4. Sascha August 28th, 2010 10:27 pm

    “I’m a UNIX hacker and I have Chinese food in my beard.”

    LOL! I just wet my pants because I was loughing so hard.

  5. Russ August 28th, 2010 10:33 pm

    Well done elan, another blog with very useful info. You rock!

  6. Twan August 28th, 2010 11:18 pm

    Can’t wait >< ‘ ! Yaaaaaaaaarh ! Seems to be great great great work out there, thanks a lot from belgium guys :)

  7. JPer August 28th, 2010 11:23 pm

    You’re still goin strong Elan! 3 more days and 3 more posts and you’ve effectively posted an update every day until release day!!!

    This post was great, very enlightening. I cant wait to take the new media manager for a spin! Thanks for everything!!

  8. MTI August 28th, 2010 11:31 pm

    Another awesome read, thanks!

    I have another question:

    What iTunes/M4V atoms are supported in 0.9.0 and what atoms do you plan (if any) to support later?

  9. elan August 28th, 2010 11:36 pm

    @MTI: We only support M4V atoms in the TV scanner at the moment, so we just use it to pick up show/season/episode when files aren’t named nicely. Easy to add the atom-reading code to an agent, just haven’t gotten around to it yet.

  10. 41john August 29th, 2010 12:30 am

    Great blog again today, so are you doing a blog a day until release… that would be awesome.

  11. zAo August 29th, 2010 12:37 am

    But I’m a UNIX hacker and I have Chinese food in my beard. Command lines FTW! The Plex Media Scanner can actually be run from the command line and do a variety of things like scan, refresh, list sections, add new sections, etc.

    Ow yeah! Is it September already? :)

  12. xain09 August 29th, 2010 1:04 am

    Funny guys! Now bring it on :)

  13. Adam August 29th, 2010 2:41 am

    How does the new Plex Media Server impact those of us who store our media on a SMB accessed non-Mac server, but view it on multiple Macs with Plex? Do we still need to employ various userdata folder tricks to share the database, or will the optimal media routing know how to tell each Mac running Plex where to find the media?

  14. Billy Joe August 29th, 2010 3:14 am

    I should also say that scanning anime series (with absolute numbering, no seasons) works perfectly now too in plex .9, all of my anime is scanning perfectly on the latest alpha without me having to change the filenames at all. theres really no standardized episode numbering format for anime releases, and the filenames can be pretty ugly, but all my 80 or so series scan the episodes perfectly now without modification. (and the tvdb agent works surprisingly well for them too)

  15. GSX August 29th, 2010 3:21 am

    Plex 9 is looking great. Awesome work. I’m also happy to see that TV shows using the VIDEO_TS format will get some attention(if not some love). I’ve really been hoping for this in Plex 9.

  16. Julien Angeli August 29th, 2010 3:27 am

    Elan, I remember in a past blog note you mentioned the pause/resume function through different frontends from a unique backend (aka I start the movie on TV A, stop it in the middle, go on the upper floor, and resume the film playing on TV B). That’s pretty convenient!

    But will Alexandria be able to stream 2 different movies at the same time? (aka movie A on TV A, and movie B on TV B upstairs for example?).

    By deploying Plex on a wider ecosystem with 1 Media Library and several frontends, I guess you will have to make Plex multi-users friendly (individual settings/ratings and logs, maybe password for some directories or just login prompt?).

    The way Plex will/would/could deal with multi-users is unclear for me. Could we have some infos about this?

    Thanks!

  17. Julien Angeli August 29th, 2010 3:37 am

    PS. I wish I could lick my own nose like Barkley. I am already drooling at Plex 9…

  18. DavidNC August 29th, 2010 4:20 am

    The most entertaining blog post I’ve read today.

  19. Ron August 29th, 2010 5:07 am

    Elan, I love your humor throughout your posts!
    I’ve tried previous plex versions and have been disappointed in the ease of use and moved to boxee.
    With Plex 9 coming out, I’m very excited to see how it
    functions now! I’m really excited about the new media scanner and its future additions.

    Sept 1st can’t get here soon enough!

    Keep up the good work!

  20. Aaron August 29th, 2010 5:30 am

    Thanks for all of the Q&A, I thought I’d attempt to join in on the action…

    Q: My keep all of my media on a Linux file server that I currently NFS mount on my Mac Mini. I love the client/server model you’re moving to, but any chance of running the Plex Media Server on Linux? I’m not sure I can afford to build a OS X based server, and I don’t want the noise of hard drives in my living room.

    Q: I don’t have a keyboard or mouse on the Mac Mini on my TV. Can I install the Plex Media Manager as a client on my Mac laptop and manage my Mac Mini’s Plex Media Server remotely?

    Thanks to Elan and all of the Plex team for all of your hard work!

  21. nick August 29th, 2010 6:25 am

    Gr8 to see! : ) I have donated some cash and see my dream will come true!!!

  22. crumbete August 29th, 2010 6:48 am

    Since we can’t specify aired order vs. dvd order right now, which does it default to?

  23. Michael August 29th, 2010 7:42 am

    It sounds like the new media scanner might finally recognize my DVDs of The Guild (watchtheguild.com). Yay!

    It’s not clear though whether this brings us closer to being able to browse TV series on DVD as a list of episodes, independent of which disc they’re on. Does it?

  24. Jack August 29th, 2010 7:49 am

    Hi Elan,

    Fantastic blog entry once again and Plex 9 is looking even more amazing then before. The turbo hashes are a work of genius.

  25. Brandon August 29th, 2010 7:58 am

    Awesome to see Plex moving forward. You’ve come a long way since the XBMC fork! Congratulations and thanks are in order.

    One question elan: When you say DVDs need to be accessible over the “local” filesystem, AFP/CIFS would qualify, correct? All my DVDs are stored as ISOs on a ReadyNAS via AFP. Please tell me this will work when I finally get my Mac Mini! :-D

  26. PJ August 29th, 2010 10:45 am

    Will the Movie and TV sections integrate content on streaming services like Netflix on demand? Someone had devised a plug-in to integrate this but will the same plug-in work for 0.9 or (ideally) is this already integrated?

  27. elan August 29th, 2010 11:54 am

    @Brandon: Any sort of network protocol works, as long as it looks like local file access to your Plex clients.

    @Gerd: We don’t have subtitle/language flags in there at the moment, likely addressing this in a subsequence release.

    @PJ: We’re focused on local content for this release, but we do have plans to allow this sort of thing in the future.

    @crumbete: It defaults to aired order.

    @Aaron: I’m sorry to say no to both of your questions, at least for the first release.

  28. Alejandro Cardenas August 29th, 2010 1:14 pm

    DUUUUUUUUUUUUDE i wanna marry you, man.

  29. DRmasSEUSe August 29th, 2010 4:07 pm

    Ok.. That’s it! I’m donating. How much do you want?

  30. Ron August 29th, 2010 4:14 pm

    First time on this one: I’m more eager for a non-apple release than I am for the official Apple new product releases (on same day). Many thanks to all who are on the dev team. Beer money via Paypal just.sent

  31. crumbete August 29th, 2010 4:55 pm

    @elan Aired order, eh? I assume that’s something that’ll be fixed in the future… but it’s kind of a dealbreaker for me for this release. Which stinks, I was pretty excited. I appreciate being informed before I installed it thought.

  32. Russ August 29th, 2010 7:11 pm

    What’s wrong with aired order? I have had zero problem with aired order, and I have at least 50 TV shows along with hundreds of episodes in total. Plex/Nine rules all :) Surely that cannot be a deal-breaker

  33. crumbete August 29th, 2010 7:46 pm

    @Russ Off the top of my head, it makes firefly play in an order that completely doesn’t make sense, because fox screwed the pooch on it. and i know i have at least 1 other show in my library that had a problem with aired order, although i can’t for the life of me remember what it was. I’ll stick with plex 8 til this gets fixed.

  34. Boots August 29th, 2010 8:21 pm

    Elan – will this work?

    A NAS hosts all the media PLUS the Alexandria database. My MacPro adds all the media to the database but is not involved in hosting the database. My iMac plays the media using the database that’s on the NAS.

    Boy – THAT would be AWSOME!!!

    James

  35. elan August 29th, 2010 9:39 pm

    @Boots/James: Hmm, sounds like you’d need PMS for your NAS. Which doesn’t exist. Today.

  36. Boots August 29th, 2010 10:22 pm

    Spoken with the tone of someone who knows >exactly< what they're doing.

    My NAS is a Netgear ReadyNAS so I doubt PMS will run on it… you guys are good but there's lots of NAS hardware out there to support! I have all the Macs at home set up mounting the NAS shares with AFP/autofs. They show up as a folder somewhere in the file system. I'm hoping I'll be able to wangle it that way – but it sounds like you have a better way planned anyhow. Ahhh dang it! Hows about you just send me the installer!!! Pretty Please?

    James

  37. Ajgoyt August 29th, 2010 11:58 pm

    Will the current plex just update for Alexandria or do we have to DL separate?

  38. elan August 29th, 2010 11:59 pm

    @Ajgoyt: New download since it’s a new series.

  39. Ajgoyt August 30th, 2010 12:02 am

    Thanks 12:00 PST or?

  40. crumbete August 30th, 2010 5:33 am

    elan, one last question and i’ll quit harping… is the aired vs. dvd order toggle planned on being added into the second release of plex 9, or is it further down the line? i’m really interested in 9, especially with the iOS app coming, but i’m happy to stick with 8 for a bit longer if i know that the feature i need is being added to 9 in the near future.

  41. Christo August 30th, 2010 6:43 am

    Great news! Keep up the excellent work!

  42. Pascal August 30th, 2010 7:20 am

    I don’t know if you do this already, but I’d really like to have the metadata right in my video files. I know it’s quite a bunch of work to sync files Alexandria but when switching to another system–without Plex but the capability the read this metadata–it would really kick ass. A while ago I considered converting all my files to use mkv as container which would allow pretty much anything about metadata but I guess I had no software to actually use anything new mkv offered (compared to mp4 or even avi).

  43. Nes August 31st, 2010 4:09 am

    Just to clarify. The server does not stream content to the client right? The server hands of a pointer to a media file to the client and the client opens the file (from a network location if necessary)?

    Great stuff this anyhow and thank you for all your hard work.

  44. Jim August 31st, 2010 9:26 pm

    Why not go with standard UPnP instead of your own HTTP based network protocol for the Plex Media Server?

    If you went with standard UPnP then we could use Plex Media Server and Media Manager with the Xbox360, PS3, NMT Popcorn Hour and all other UPnP clients out there :)

  45. elan August 31st, 2010 9:35 pm

    @Jim: In a word, metadata. I don’t think UPnP has rich enough metadata. There are other reasons, but that’s the biggest one in my mind.

  46. elan August 31st, 2010 9:37 pm

    @Nes: The Plex Media Server streams content to the iOS client. Plex for Mac gets its content from the server via HTTP only if it can’t find it locally (this is Plex’s Optimal Media Routing feature).

  47. Juan September 2nd, 2010 8:50 pm

    This has been an huge change in spirit. You are getting more and more apple-izer with time: no raw folders, “many one-more-things”, surprises after long silences… But although I like it a lot (it is amazing the work you have done both in Plex and in the scanner-server, that has made me felt stupid having bought “air video” some weeks ago :/

    But at the end i miss the “Delete item” I had in contextual menues, since it helped me keep my disk in shape.

    Is there any way to have it back?

    Tx. (I am absolutely amazed)

    -juan

  48. badmash October 22nd, 2010 8:21 am

    I just signed up to your blogs rss feed. Will you post more on this subject?

Leave a reply