Fixing Letterboxed and Pillarboxed Transcoding in MythTV

Posted to linux/mythtv on 2007-09-05 10:08:00

The recent demise of Zap2it labs and the subsuquent creation of Schedules Direct was a pretty remarkable change in the MythTV community. The community now has an almost official blessing from a TV Guide service to continue, a pretty big event for most open source projects.

However, due to the infrastructure of MythTV, it's not easy to add a new TV listing service. Rather, the change from Zap2it to Schedules Direct required all users to upgrade their copies of MythTV to the newest version of the stable branch. Unfortunately, this version still has issues with transcoding high definition signals. The major issue is that mythtranscode has no knowledge of the program aspect ratio. It ends up storing pixels that aren't needed, such as what happens when a 16:9 signal is sent in a 4:3 signal (letterboxed) or when a 4:3 signal is sent in a 16:9 signal (pillarboxed, much more common). In an ideal world, MythTV would know how to cope with these automatically when transcoding and just magically cut off the offending portions.

A pillarboxed episode of The Simpsons
Pillarboxing on an episode of The Simpsons recorded from a high definition source
A letterboxed episode of Frontline
Letterboxing of a Frontline episode recorded from a standard definition source

While the result is still acceptable, it's not perfect and may force you to switch aspect ratios to watch the video. About a year ago, I wrote a patch for MythTV that transcoders to specify a target aspect ratio. This was against MythTV 0.20, and because the newer versions didn't provide that many desirable features, I never updated the patch or my installation of MythTV. Facing the impending loss of guide data just days before that start of the NFL season, and only weeks before the start of the new TV season, I updated the patch for MythTV 0.20.2. You can find the transcoding aspect ratio patch in the MythTV Trac system.

The patch still is far from perfect, and you'll need some cron hackery to get it to work right. The main reason is because although MythTV allows you to create non-standard transcoder names in the frontend, it won't let you use them for anything but the default transcoder. To get around this, you'll need to look into your MythTV database and get the transcoder id numbers. For my system, I created a set of transcoders for 1080i, 720p, and 480i content that give me the option of whether or not to resize on the transcode and the quality. Run the following command on your database to get a listing of the transcoders:

mysql> select * from recordingprofiles;
+----+-------------------+-------------------------+-------------------------+--------------+
| id | name              | videocodec              | audiocodec              | profilegroup |
+----+-------------------+-------------------------+-------------------------+--------------+
                                          :: snip ::
| 27 | 1080i No Resize   | MPEG-4                  | MP3                     |            6 | 
| 28 | 1080i Resize      | MPEG-4                  | MP3                     |            6 | 
| 29 | 720p No Resize    | MPEG-4                  | MP3                     |            6 | 
| 30 | 480i No Resize    | MPEG-4                  | MP3                     |            6 | 
| 31 | 720p Resize       | MPEG-4                  | MP3                     |            6 | 
| 32 | 720p Resize Med   | MPEG-4                  | MP3                     |            6 | 
| 33 | 1080i Resize Med  | MPEG-4                  | MP3                     |            6 | 
                                          :: snip ::
| 46 | 1080i SDTV Resize | MPEG-4                  | MP3                     |            6 | 
| 47 | 720p SDTV Resize  | MPEG-4                  | MP3                     |            6 | 
+----+-------------------+-------------------------+-------------------------+--------------+
47 rows in set (0.01 sec)

Transcoders 27, 28, 29, 31, 32, and 33 all leave the aspect ratio unchanged. Transcoder 30 is a transcoder that resizes 480i letterboxed content. Transcoders 46 and 47 remove pillarboxes from SDTV signals transmitted over HD. You'll also notice that I have different transcoders for 1080i and 720p. You don't need to do this, but I have chosen to do this to avoid deinterlacing 720p signals -- saving a bit of time and power.

The final step is to tell MythTV to periodically update the transcoders settings. I use a simple script called from cron about every 15 minutes:

#!/bin/sh
MYTHPASSWORD="-pYOURMYTHPASSWORD"
echo "update recorded set transcoder=28 where chanid=3215 or chanid=3220 or chanid=3212 or chanid=3211;" | nice -n 10 mysql -u mythtv $MYTHPASSWORD mythconverg
echo "update recorded set transcoder=31 where chanid=3213 or chanid=3210 or chanid=3214;" | nice -n 10 mysql -u mythtv $MYTHPASSWORD mythconverg
echo "update recorded set transcoder=47 where (chanid=3213 or chanid=3210) and (title='The Simpsons' or title='Family Guy' or title='American Dad' or title='King of the Hill');" | nice -n 10 mysql -u mythtv $MYTHPASSWORD mythconverg
echo "update recorded set transcoder=47 where (chanid=3214) and (title='Mad Max' or title='Bubble Boy' or title='Hotel Rwanda');" | nice -n 10 mysql -u mythtv $MYTHPASSWORD mythconverg
echo "update recorded set transcoder=47 where (chanid=3210) and (title='The Ten Commandments');" | nice -n 10 mysql -u mythtv $MYTHPASSWORD mythconverg

This script forces the 1080i channels to use transcoder 28 and the 720p channels to use transcoder 31. The last three lines are for pillarboxed programs that recorded off 720p that must be unpillarboxed. With this in the background, when transcoding I select the default transcoder and the output will be properly cropped.

As an added bonus, this patch also fixes issues with mythtranscode not understanding that 1080i signals are encoded as 1088 (due to MPEG files needing to be a multiple of 16 lines high) and encoding a nasty gray bar at the bottom of your 1080i transcoded vidoes. It also cuts off the top eight pixels of most recordings because Comcast normally sends junk on those lines that looks like static.

Update: I didn't mention on my original post that if you schedule a recording via MythWeb, there is an option to set the transcoder. In this screenshot (click for full size), you can see where I specified the transcoder "720p SDT Resize" for use on "The Simpsons".

Setting the transcoder in Mythweb.  Click for full size

Measuring Power Consumption in MythTV

Posted to linux/mythtv on 2006-08-08 16:42:00

Somehow I'm becoming a bit of a treehugger. Either that or I'm just getting annoyed with how hot rooms in the house get as a when I have computers on all the time. So, I set out to figure out how much power each of my computers takes to run and how much this power costs. The easiest way way to measure these devices was with my handy Kill A Watt. A rather nerdy Christmas present from Kristina a few years ago (I asked for it).

To test each of the machines I hooked them into the Kill A Watt for an extended time period and watched the load of the machine at different times. For multiple tasks on the machines I would read the Watts from the Kill A Watt and then look at the overall power usage over an extended period. For the costs of electricity I used the price on my July 2006 statement from Duquesne Light, which is about $0.10 per KWh. At this rate, 1W on all year costs a little under a dollar.

So let's start out with dearly departed Scissors. Unfortunately, I did the measurements on scissors a long time ago and since have given it away, so I couldn't get the same variety of measurements. Scissors was a tiny K6-200 machine with 96MB of ram and a single Quantum Fireball 4.3GB hard disk. It had no other disk drives, furthermore, the fans on the machine were broken. Over a course of 100 hours, this machine consumed about 3.78KWh - an average usage of 38W, for a yearly cost of $32.58.

Next I monitored the power consumption of Igel, my current server machine. This machine is a PIII-900MHz machine with a single 45GB drive, CDRW drive, and 768MB of RAM. As this is a server, I didn't break the load down by category. Over 1222 hours (yes, that's well over a month), the machine used 84.4KWh of power, for an average consumption of 69W and a yearly cost of $59.22. The faster machine costs nearly twice as much as the older machine to power.

Next up I did two rounds of measurements on Spongebob, my MythTV box. The first round of tests were done before the upgrade I gave it as a birthday present to myself. At this point, Spongebob was an AMD Athlon 64 3500+ with 3 200GB hard disks, 1G of ram, a WinTV PVR 250, a pcHDTV HD-3000, and a DVD drive. I should also mention that the northbridge fan was not working. Over 96 hours the machine used 10.17KWh of power, for an average of 106W and yearly cost of $90. When watching HDTV the consumption shot up to 130W - increasing the cost to $111. Luckily, it's not watching HDTV that often.

Recently I upgraded Spongebob to a AMD Athlon 64x2 4400, 1 200GB hard disk, 4x320GB hard disks, 1G of ram, a WinTV PVR 250, 2 pcHDTV HD-3000s, and a DVD drive. I also replaced the northbridge fan with a really fancy lanboy style cooler. Initially, however this new configuration did not support CPU throttling. The load was averaging about 165W making the cost $142.32. Ouch. That had to change. Luckily, there's a way to fix this that involves hacking your kernel and your ACPI settings, but I'll cover that later. When monitored over 97 hours the machine consumed 13KWh of power, for an average of 134W and a cost of $114.89. Watching HDTV put the usage back up to 165W, which left the $142.32 cost. Overall, adding in the upgrades costs me about $25/yr in power costs, significant, but still the MythTV box comes out to less than $10/month.

For convenience, here's a chart showing some of the power costs:

Machine Avg W/h Yearly Cost
Scissors 38 $32.58
AMD K6 200, 4.3GB disk, 96MB ram
Igel 69 $59.22
PIII 900, 45GB, 768MB ram, CDRW
Spongebob 106 $90.83
Athlon 64 3500, 3x200GB, 1GB ram, DVD
Spongebob 134 $114.89
Athlon 64x2 4400, 1x200GB, 4x320GB, 1GB ram, DVD

So what's this going to change? I had no idea that running Igel was costing me $60/yr for just power. Wow. That's gonna make me do one of two things, either migrate the virtual machines down to the MythTV box, or purchase a cheapo shared server -- probably the former. With that sort of change, our power consumption will be under the levels from when I had scissors as my server and the pre-upgraded Spongebob as MythTV (total consumption of 141W) as opposed to 134W now. Still, I'm a little shocked at the power consumption of everything. I wish I had an easier way to quantify the power consumption of individual components of the system. If anyone knows of a way, short of ramming probes into the system, I'd love to hear it.


Faster Transcoding on AMD Athlon 64's

Posted to linux/mythtv on 2006-01-13 11:18:00

If you're running MythTV on an Athlon 64 platform, you've probably noticed that the CPU throttles itself down when transcoding and cutting commercials despite the fact that your processor may show as being 100% utilized. This results in much slower transcoding than one would normally expect, but saves a small amount of power.

However, if you record a lot of stuff, or even a moderate amount of stuff in HD, this property and increased delay can be quite annoying. It can take several extra hours to cut and transcode an hour long HDTV program when running at 1.2GHz instead of 2.4GHz (my analysis shows a greater then 2.5x slowdown - which is weird in itself). First, let's cover the reason you system slow down, and then we'll cover what to do about it.

Processors take a lot of power - most of that power is dissipated as heat because of the resistance in the CPU. Slowly, AMD and Intel are realizing that people don't want extra heaters in their living room/office, that powering all of these computers puts a real dent in your wallet, and that most of the energy is wasted because the processor is idle anyway. The solution? Create not only more effecient processors, but also allow processors that throttle back to conserve power when they're not being fully utilized. On the Athlon 64 platform this is called "Cool'n'Quiet" and has been available since the Winchester core chips.

Cool'n'Quiet works with a couple of different parts: a Cool'n'Quiet enabled processor, a Cool'n'Quiet enabled motherboard, and a software program that manages it all. If you're running a socket 939 Athlon 64, you've almost certainly got the first two. Under Linux, you can directly control the CPU power state through the proc file system, or you can have a daemon process, such as powernowd, manage it for you. Ubuntu installs powernowd automatically and starts it for you, so out of the box you're getting some of the benefits of this technology.

However, powernowd is configured to not take into account nice processes. Normally, this is fine, nice processes can afford to run a bit slower, that's why they're nice in the first place. Unfortunately, it is only set up to consider nice processes as a group and can't differentiate between a process that can't take advantage of increase speed and one that can, such as transcoding. And, as you've probably guessed by now, to minimize interference with other programs, MythTV runs transcoding and commercial flagging processes as nice. To speed them up all we need to do is tell powernowd to take into account nice processes.

In Ubuntu, this is really straight forward. Most startup scripts for daemons look in the /etc/default directory for local configuration overrides. By default powernow is run with only the -q flag, telling it to be quiet. If we also pass the -n flag, it will take nice processes into account when deciding whether or not to throttle the processor. The solution, create a file /etc/default/powernowd with a single line in it:

OPTIONS="-q -n"

Then issue the command /etc/init.d/powernowd restart and your computer will no longer throttle itself when transcoding and commercial flagging. Enjoy the increased speed of transcoding and happy MythTVing.


Electromagnetic Lovin

Posted to linux/mythtv on 2005-11-20 21:20:00

After much frustration I've decided that no matter how many bells and whistles the MSI K8N-Neo4 Platinum has, it still can't make up for the fact that I get some crazy bad electormagnetic interference when the system is under heavy load. At first I thought it was just going to be on the pcHDTV HD-3000 card, but today during the Steelers game, while I was watching the game while it was recording and had commercial flagging running and the MPEG file just started to crap out. Then blammo, MythTV died. This is really annoying. It doesn't appear to actually mess up data on the bus, just the ability of the tuners to pull in channels using internal tuners.

After three weeks of fighting, vengeance will be mine. I'm returning the board and getting a new ASRock 939Dual-SATA2 board instead. This requires me to pick up a firewire card too and means that all the slots will be filled up from the get go. But whatever, we'll see how this works. I may have to do another return if that doesn't work well.


MythTV, Ubuntu, Athlon64, WinTVPVR, Comcast Digital Cable, and a pcHDTV HD-3000 in a pear tree!

Posted to linux/mythtv on 2005-11-17 11:08:00

I've been writing a nice tutorial on setting up MythTV on Ubuntu x86_64 over the course of the past few weeks. Please feel free to browse it and give feedback. It still is a work in progress and I could certainly use some help in some spots. I should clarify this by explaining that I've got my system more or less working, with the exception of static on my pcHDTV which I understand is an electro-magnetic interference issue that I can't do anything about. When I move that card into another computer, I'll add details about that too.

For those that are concerned, I'll eventually put up a TRAC site for managing the project and and providing access to the repository of revisions. My goal is to provide a slightly more up-to-date and Ubuntu centric version of Jarod Wilson's Fedora MythTV Guide. I know, that's a lot of hoping. Come on magic google juice, lead people to me.


Motorola DCT 6200 Firewire Workaround

Posted to linux/mythtv on 2005-11-10 18:48:00

I wrote a patch for MythTV that fixes some of the issues with a DCT 6200 being on a system where node 1 is not established. This is useful for folks like me have multiple firewire ports on their PCs. I feel ownership in MythTV now. Woot.


DVB and Digital Static

Posted to linux/mythtv on 2005-11-09 08:32:00

It kills me how I'm so close I can feel it, but it's completely out of my power to reach a 100% working system right now. The reason? I'm experiencing digital noise on my HDTV tuner card. This REALLY blows. When the system is at low load, I have no problem recording HDTV programs, however, as the load increases, I see lots of noise show up on my system. Apparently, I'm not the only one having these problems, as this thread from the MythTV users list shows. I've also started a discussion on the pcHDTV.com forums.

So what's a guy to do then? I bought a system to make it HDTV compatible, but the HDTV is sucking it up. I can get HDTV over the firewire, so I may just have to loose digital cable for right now. Maybe this is the kick in the ass that I needed to create the mutually exclusive tuners. Here's a brief explanation of what this would do.

Normally, I've got my WinTV PVR-250 hooked up to the cable box via svideo, and the channels are changed through firewire. This lets me record all the channels out of the cable box in analog mode. This card is also hooked up to the standard cable input and can tune the lower channels just fine. In addition, I'm able to pull the firewire HDTV signal from the box onto my computer. However, as you can now see, when I pull in the firewire HDTV signal, I'm going to end up changing channels on the WinTV PVR-250 that is getting it's signal over firewire. So, I'd like to make it so only one of either the WinTV PVR-250 (with SVideo) and the Firewire cable input can be used at the same time. Furthermore, it would be neat if as an extension, when the Firewire input was in use, it would automatically let my WinTV PVR-250 know that it should use the standard cable tuner input. I wonder how hard this would be to hack together.


My MythTV Box Blew Up

While not entirely true, that's the gist of what happened. Last Sunday, October 30th, the machine was hanging a lot. After lots of cussing, I decided just to shut the machine off and banish it to the corner for a while. When I got back the machine wouldn't post and was making a funky smell. I'm still not sure if it's the motherboard, processor, or power supply. I hope it's just the power supply.

In any case, it provided a nice excuse to upgrade my computer with new toys. All of the cards and drives inside the computer were fine. Unfortunately, I couldn't reuse much of the other components such as memory, or rather chose not to reuse much of it. Here's what I got for the new computer:

  • AMD Athlon 64 3200+ (Venice core)
  • MSI K8N-Neo4 Platinum Motherboard
  • XFX GeForce 6600 (256MB ram)
  • 1GB Ram
  • Antec Overture II Media Center Case

Since then I've been fighting to get everything working again. But it looks like it's all basically going. We were able to watch the high-definition version of Monsters Inc. with little problem. The DVD player and remote work fine. I even have it controlling the cable box so we can record digital cable or high definition right from the cable box. Tonight I'm going to get pcHDTV card working on it, provided that I can actually get XvMC to work properly. I'm working on a thorough writeup on how to get MythTV working in this sorta setup, and I'll post it soon.

In the mean time, here are some pictures of the system as I built it and put it in the entertainment center. For reference on what the old setup looked like, you may want to see these pictures: old mythtv box, old entertainment center.

Update 11/08/2005: I started writing this yesterday, but didn't donwload the photos until this morning. In the mean time I've managed to get XvMC working by beating it into submission using a brute force method. This means that HDTV is more or less working, but I can tell that my disk I/O is hurting. It may be time to invest in that large RAID 5 array soon.


Comcast Says Screw You!

Posted to linux/mythtv on 2005-09-28 15:09:00

The title sums it up quite nicely. Instead of making it easy for me to watch HDTV, they're making it quite difficult. I called them last week to inquire about HDTV support on cable. I was told that I needed to get a cable box, that would run me $5.99 a month. For another few bucks I could get the digital classic package. Not too bad I guess. Anyway, today the guy arrived. Was a little shocked to see that I didn't have a box but he had one in his truck, a Motorola DCT 6200 to be exact. This is the nicer one with firewire outputs.

Anyway, we hooked it up pretty quickly and I was getting the stations. The problem is that I didn't have any menu support, so I had no guide information. Furthermore, I couldn't actually get HDTV to display on my DVI output. That blew. Instead, I got a nice message saying that it couldn't authenticate me. What the fuck? I can barely understand this message, I could hardly imagine what would happen to grandma reading it.

What was happening is that box was trying to authenticate using HDCP to my monitor which doesn't support it. It's not like I can copy the digital signal from the monitor, but I digress. Basically, when the DVI is plugged in or powered on it waits a few seconds to get a handshake back from the monitor saying that it won't let you copy stuff. If it doesn't get it, it blacks out. So much for fair use. To make things worse, it appeared that I could only get the display when using the DVI, so I had to any settings or browsing for video on demand in the few seconds before the cable box decided I couldn't see that anymore. What the hell?

After some digging around, I found the DCT 6200 User Guide on Motorola's site. It still didn't provide any information about how to get to the uber setup screen that the tech who installed the box got to, but it did tell me that if it was set up to output HDTV then it would not be able to overlay the graphics on the SVideo or composite outputs. Wow, what crappy American engineering. There is hope, you can get to the setup by turning the box off, then hitting the menu key when the box is off. This will let you do some menu stuff using the LED displays on the front of the cable box. Basically, set all outputs to 480i and you'll be able to get your display back.

So now I decided to play with HDTV. It turns out that I can use the box to tune HDTV, but it will downgrade the coax output to SD. Bullshit. That's retarded. So, I tried hooking up my cable line directly to my pcHDTV card. Running some fun scanning software shows me that indeed I'm getting lots of signals. 221 of them to be exact. That can't be right I thought. Running the file command on a single dump didn't prove helpful, it just said data. Mplayer was choking on the first few. Still, it seemed odd. So, I decided to write a little script to scan all 221 channels. Voila, I've got some HDTV channels coming in. Fear Judge Hatchett in HD.

Now I'm left wondering if I need the box at all. I can probably just cancel it because it doesn't work as advertised. But then I would loose Sci-Fi, basically the only good new channel I get. We'll have to see. I may also decide to pick up another tuner card and a firewire cable to control the box and just record programs from that. Could be a lot of fun. More updates later.

Update: One of my helpful readers pointed me to the DVI Magic device, which is sold as an HDCP compatible DVI amplifier. Looks very interesting, but a but too expensive for me at 399 euro's. Pity.


LVM with MythTV on Ubuntu

Posted to linux/mythtv on 2005-09-27 10:19:00

Last night I was pleasantly surprised to find that my new hard disk from Outpost.com/Fry's had already arrived, despite ordering it on Friday and only paying for ground shipping. Note to the other folks from the Burgh, Outpost ships from Columbus, thus you get your stuff fast. Having received this nice shiny new 200GB hard disk, I needed a way to integrate into my system. I consider this disk to be a stop-gap solution until I can afford a RAID-5 setup.

Anyway, before I began this little adventure, I was running low on storage space. I keep my MythTV recordings in /mnt/store and the 200G drive I picked up in February was now loaded down with lots of Simpsons, Family Guy, and various movies. The overall disk picture looked something like this:

patrick@dreams:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda6              33G   24G  6.9G  78% /
tmpfs                 380M     0  380M   0% /dev/shm
/dev/hda1              90M   24M   62M  28% /boot
/dev/hdc1             190G  185G  1.9G  98% /mnt/store
/dev/hda5             9.3G  8.7G  601M  94% /home
/dev                   33G   24G  6.9G  78% /.dev
none                  5.0M  2.8M  2.3M  56% /dev

In addition, I have an NEC DVD dual layer burner on /dev/hdb. My task at hand was to merge the new 200GB disk with the existing 200GB disk. I had a variety of possible tools for this, including just a bunch of symlinks to the old files, unionfs, and LVM. Symlinks might work okay, but there is a lot of manual work that needs to be done. On the bright side, if I lose a disk with that method, I only lose the recordings on that disk. UnionFS allows me to ignore the symlink problem and stack the two drives. However, one drive would be read only, so it would never get any more full. This really wasn't desirable either. Really, I need some sort of RAID setup. Not so much for the redundancy, but for the consistent disk image across drives.

So normally what I'd be looking for is called RAID-0 or JBOD (Just a bunch of disks). The difference between the two is that RAID-0 stripes the data between the disks, meaning that different chunks of the data are on different disks. This gives increased I/O throughput provided that the disks are on different controllers. JBOD just concatenates the disks together as one big disk and when the first disk fills up, work begins on the second disk. The problem is that both of these would require me to blow away both drives to get going. I'm not about to try and find all those Simpsons expisodes again. Furthermore, if I ever wanted to add more disks to this setup, I'd still have to go this pain again. This was not desirable.

Enter the joy of LVM for Linux. LVM stands for Logical Volume Management, and it's a method that you can be completely agnostic about the underlying disk setup for a mount point. The current version is LVM 2 and most distros ship with LVM2 support enabled (Ubuntu works just fine with LVM). The basic concept with LVM is that at the top level you have the concept of a volume group. This is a group of disks that you can create many mount points out of. For most users, you'll be fine with just one volume group, but if you're going to be doing a lot of moving of disks, you might want to have different volume groups to make things easier. Within a volume group you have a collection of physical volumes, which are essentially just block devices. Under Linux this means that you can either dedicate an entire disk or just a partition of a disk. In my case, I was dedicating two entire disks to LVM. This means that I get an extra 1K of space by not needing the partition table at the front of the disk. Woohoo. So, my first step was to initialize the new hard disk, now called /dev/hdb as an LVM physical volume.

root@dreams:~# pvcreate /dev/hdb

With an initial physical volume created, it's possible to move on and create a volume group. Like I stated earlier, for most home users, having a single volume group will be just fine. Especially if you're just using it for MythTV. To get things started, I created a volume group called mythtv_store to indicate that I was using it for MythTV and told LVM that I would be using /dev/hdb as the first physical volume in the group. You'll also need to tell LVM that the volume group is accessible. This is what the second command does.

root@dreams:~# vgcreate mythtv_store /dev/hdb
rootodreams:~# vgchange -a y mythtv_store

The final thing that you'll need if you want to access your disk is to create a logical volume on top of the volume group. It's possible to use LVM to set up all sorts of fun stuff like striping between disks, but we're not going to be doing that because we're starting with just one disk. We want the setup to take up the entirety of the disk currently allocated to the volume group. To do that we first figure out how many physical extents we have in the volume group. A physical extent is a chunk of a physical disk that can be allocated to a logical volume. When we run the vgdisplay command (more about that later), we'll how man physical extents the drive has by looking at the line that says Total PE. For convenience and confusion sake, we'll call our new logical volume mythtv.

root@dreams:~# vgdisplay mythtv_store | grep "Total PE"
  Total PE              47695
root@dreams:~# lvcreate -l 47695 mythtv_store -n mythtv

If you're interested in seeing what's going on you can run pvdisplay to show the physical volumes, vgdisplay to show the volume groups you've just created, and lvdisplay to show the logical volumes. Here's the output from my system (somewhat faked as I'm writing this after creating the LVM setup):

root@dreams:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/hdb
  VG Name               mythtv_store
  PV Size               186.31 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              47695
  Free PE               0
  Allocated PE          47695
  PV UUID               e5obta-U4u1-gdWB-DwGP-MTBo-Lpw5-1O7rVK

root@dreams:~# vgdisplay
  --- Volume group ---
  VG Name               mythtv_store
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               186.27 GB
  PE Size               4.00 MB
  Total PE              47695
  Alloc PE / Size       47695 / 186.27 GB
  Free  PE / Size       0 / 0
  VG UUID               2uX3MD-1386-FqXx-LMEJ-ELOW-nB58-UNEoTl

root@dreams:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/mythtv_store/mythtv
  VG Name                mythtv_store
  LV UUID                CbjYXs-gWnC-63X6-ED2D-x3BS-PbkC-td0OT4
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                186.27 GB
  Current LE             47695
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

The first line of the lvdisplay output is what we're looking for. This is the device name that you'll use to access the logical volume, in my case it is /dev/mythtv_store/mythtv. In general it goes by the name of /dev/[VOLUME GROUP NAME]/[LOGICAL VOLUME NAME]. From here we can see that we've got a 186.27 GB logical disk that we've just created. The next step is to format the thing and port our data over. I've been using JFS because it has speedy deletes for the large files that MythTV produces, however, you may want to consider using EXT3 or ReiserFS. The main reason is because you cannot shrink a JFS drive (or XFS for that matter). If you want a filesystem you can shrink, you must use EXT3 or ResierFS (at least at this time). Anyway, here's how I formatted the drive, mounted it, and copied all my data from the old drive over (the old drive was mounted at /mnt/store).

root@dreams:~# mkfs.jfs /dev/mythtv_store/mythtv
root@dreams:~# mkdir /mnt/tmp
root@dreams:~# mount /dev/mythtv_store/mythtv /mnt/tmp
root@dreams:~# cp -a /mnt/store /mnt/tmp

A quick look shows that all my files copied successfully. If you haven't already, now would be a good time to shut down MythBackend, but lets hope you did that a long time ago. Now comes the part that you'll feel queasy about, but it should work. We're going to blow away the old disk, add it to the volume group, and then extend that logical volume and our JFS file system. The first thing that we need to do is blow away the partition table on /dev/hdc. This is necessary because LVM doesn't like seeing a partition table there if it's going to use the whole disk. Thanks to dd this is pretty straight forward.

root@dreams:~# dd if=/dev/zero of=/dev/hdc bs=1k count=1
root@dreams:~# pvcreate /dev/hdc
root@dreams:~# vgextend mythtv_store /dev/hdc

These steps have added /dev/hdc to the volume group, but it hasn't yet been added to the logical volume. To do that we're going to need to know the total size of volume group. Once again, grep becomes our friend, and we use the following commands:

root@dreams:~# vgdisplay mythtv_store | grep "Total PE"
  Total PE              96315
root@dreams:~# lvextend -l 96315 /dev/mythtv_store/mythtv

Now if we run lvdisplay we'll see that the disk has been extened. However, looking a df it's clear that the disk space has not been increased. This is because we haven't told JFS to extend the disk yet.

root@dreams:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/mythtv_store/mythtv
  VG Name                mythtv_store
  LV UUID                CbjYXs-gWnC-63X6-ED2D-x3BS-PbkC-td0OT4
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                376.23 GB
  Current LE             96315
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

root@dreams:~# df
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda6              33G   24G  6.9G  78% /
tmpfs                 380M     0  380M   0% /dev/shm
/dev/hda1              90M   24M   62M  28% /boot
/dev/mapper/mythtv_store-mythtv
                      186G  185G  739M  99% /mnt/store
/dev/hda5             9.3G  8.7G  601M  94% /home
/dev                   33G   24G  6.9G  78% /.dev
none                  5.0M  2.8M  2.3M  56% /dev

The final step is to tell JFS to resize the mounted filesystem. In order to do this, JFS needs to execute a remount/resize command. This means that you need to have the filesystem mounted, then execute the following command:

root@dreams:~# mount -o remount,resize /mnt/store

If all goes well, it will take a few seconds to mount the disk, and then you'll have a nice big disk to store all of your MythTV recordings on. Here's the final output from the important commands on my system:

root@dreams:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/hdb
  VG Name               mythtv_store
  PV Size               186.31 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              47695
  Free PE               0
  Allocated PE          47695
  PV UUID               e5obta-U4u1-gdWB-DwGP-MTBo-Lpw5-1O7rVK

  --- Physical volume ---
  PV Name               /dev/hdc
  VG Name               mythtv_store
  PV Size               189.92 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              48620
  Free PE               0
  Allocated PE          48620
  PV UUID               0poKpC-804L-kWvX-D97a-23Fc-L4UT-6HWObk

root@dreams:~# vgdisplay
  --- Volume group ---
  VG Name               mythtv_store
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               376.23 GB
  PE Size               4.00 MB
  Total PE              96315
  Alloc PE / Size       96315 / 376.23 GB
  Free  PE / Size       0 / 0
  VG UUID               2uX3MD-1386-FqXx-LMEJ-ELOW-nB58-UNEoTl

root@dreams:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/mythtv_store/mythtv
  VG Name                mythtv_store
  LV UUID                CbjYXs-gWnC-63X6-ED2D-x3BS-PbkC-td0OT4
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                376.23 GB
  Current LE             96315
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           254:0

root@dreams:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda6             33645952  24729060   7207756  78% /
tmpfs                   388228         0    388228   0% /dev/shm
/dev/hda1                91599     24143     62569  28% /boot
/dev/mapper/mythtv_store-mythtv
                     394461228 179175688 215285540  46% /mnt/store
/dev/hda5              9732200   9116892    615308  94% /home
/dev                  33645952  24729060   7207756  78% /.dev
none                      5120      2856      2264  56% /dev