Troubleshooters.Com and T.C Linux Library Present
Converting .ra Files to .ogg
Copyright (C) 2004 by Steve Litt
[ Linux Library | Troubleshooters.Com | Email Steve Litt | Copyright Notice ]

CONTENTS:

Why?

RealAudio is not free software. Source isn't easily available. As time continues, RealAudio files may become increasingly unsupported under Linux. In fact, I've found it increasingly more difficult to run realplay under Linux. The binaries distributed by the Real folks haven't been updated for years, and are hard to find. Linux libraries and kernels keep evolving, so it's possible that sooner or later a modern Linux distro will NOT be able to run RealPlayer under any circumstances. It's also possible that the Real folks will stop distributing the RealPlayer binaries.

Other .ra conversion techniques exist, but at least one was laid low by DMCA type legislation. The vsound project was abandoned by its originator for legal reasons, as described on http://www.mega-nerd.com/erikd/vsound/...

October 27th, 2002

Although I have maintained vsound for nearly three years, I can no longer do so, nor can I continue to make it available from this web site.

I live in Australia which has a law (Digital Agenda Bill 2000) which is similar to the DCMA in the US in that it makes the distribution of a devices for circumventing copyright protection illegal. I have neither the time, money or inclination to make myself a possible target for such legal action by companies with endless legal and financial resources.

However, vsound is probaly available from other web sites. If you want a copy, you should search the web. Do not email me as I cannot and will not provide you with a copy.

Erik de Castro Lopo.

Even if RealAudio is perpetually supported under Windows, conversion under Windows might be problematic.

As states pass "Super DMCA" laws, even the most innocuous format translations of files or content you legally own might become illegal. If "Digital Rights Management" becomes the law of the land, such innocuous format translations could become technologically impossible without dabbling in the dangerous and illegal world of black market contraband equipment and software.

The bottom line, if you have important .ra files, now might be the time to convert them to a different format, or risk losing them forever.

Be Legal or Be Gone

Do not use this document to violate the law. Convert only what you have the right to possess. NEVER redistribute audio not created by yourself.

To Whence?

What should be the target new format? .mp3 is patent encumbered, meaning although MP3 players are free right now, they could become costly or unavailable later. If the patent is asserted, it could become unavailable in Linux. Windows Media Player format is proprietary -- not a good repository for your important data. Of all alternatives, .ogg files offer the best future. Free software from the GNU people, .ogg files are guaranteed patent free. .ogg players and recorders cannot be removed from Free Software unless the government passes a "digital management rights" law.

Executive Summary

This document discusses a method to convert .ra files to .ogg files. This conversion is definitely not easy or obvious. It's not the only method, or even the best method. It requires a great deal of user intervention, and some trial and error. But the result is a reasonably clean .ogg format of your original .ra file.

You should also know that the author is not an expert on either sound or RealAudio. The information in this document details how I successfully converted a .ra file. There are almost certainly better methods, but such better methods are not widely known on the Internet.

Conversion Quickstart

NOTE:

Throughout this document, all examples assume that the original .ra file(s) was encoded at 11025 samples per second.

Obtaining `realplay` for Linux

This conversion method requires realplay for Linux. If you don't have realplay for Linux, grab it at http://scopes.real.com/real/player/unix/unix.html. Install it as per instructions and installer program. I found it rather straightforward, except that you might get the mostly black window or the no-sound as detailed in the Symptoms and Solutions section of this document.

Procedure

Find out the sample frequency of the original .ra file(s)

Throughout this document we assume 11025

Set realplay to output ESound

View->Preferences->Performance_tab->ESound_Support

Run ESD

```
esd  -r 11025 -d /dev/dsp
```

Pipe and translate /dev/dsp to out.ogg:

esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o out.ogg -

Immediately play the song

~/RealPlayer8/realplay mysong.ra

When the song stops playing and the spinning stick stops spinning

Ctrl+C out of the esdmon command

Close realplay

Notes

The esd -r option, the sox -r option and the oggenc --resample option must be set to the sample rate of the original .ra.

Setting these options higher than the original might at first sound like a treble enhancement, and at first might sound pleasant. However, after listening for a few minutes you'll discover such treble boost fatiguing, because in reality it's not treble boost, but merely distortion, combined with a high frequency oscillation (ringing). What has happened is that the software has tried to create data where no data existed. This is very similar to the pixelation you get when zooming in on a low resolution image -- it's big, but so distorted as to be less pleasant than the unzoomed version.

If the resulting .ogg is too soft or too loud, adjust the sox -v option. Remember that hearing is logarithmic, so doubling the -v value increases the perceived volume by a linear amount. The final volume should be comparable to your native .mp3 files or your .ogg files ripped from CD. Please remember that if the volume is boosted too much, distortion results. If you need to lower the volume below what -v1produces, use fractional values.
If the realplayer program starts out with the majority of its window black, and it is inoperable, or if you receive an error saying "Cannot open the audio device. Another application may be using it.", or if the sound is distorted, displeasing or screachy, see the article on symptoms and solutions.

Principles of .ra and .ogg

.ra files are RealAudio streamer files designed to be played from a hard disk. Although I haven't tested it, I believe these same principles could be used to convert true streamer .ram files to .ogg.

The conversion method detailed in this document is to "play" the RealAudio file in RealPlayer, pick off the digital output, and convert that. This is a little like recording from the line out jack of your radio. You don't truly record the radio waves, but instead pick off the play of your radio.

NOTE:

Throughout this document, all examples assume that the original .ra file(s) was encoded at 11025 samples per second.

In this case, you pick off the digital audio from /dev/dsp using the esdmon program. The process looks something like this:
Diagram of the process

The .ra song is played by realplayer, which is configured with the ESound audio driver, which outputs via ESD to /dev/dsp because you have previously run a command somewhat like this:



esd  -r 11025 -d /dev/dsp

The preceding sends esd output to the /dev/dsp device. The -r 11025 specifies that it will be sampled at 11025 samples per second. It is vital that you make this number match the sampling rate of the .ra files. If the number is lower than the sampling rate of the .rafiles, you'll lose high frequencies and crispness. It it's higher than the.ra files, the computer's software will try to sample where no samples exist. At first glance this might seem a great way to enhance highs for an inadequately recorded .ra file, but what it really does is interpolate the existing, gettting some of it right and some of it wrong. This is very much like zooming in on a low resolution picture -- you don't get a better picture -- you get pixelation. In this case, if the number is greater than the sampling rate of the .ra file, you get harsh high frequency distortion coupled with a distracting resonance.

How do you set realplay to output to the ESound audio driver? Perform the following within realplay:

View->Preferences->Performance_tab->ESound_Support

If you read the prompt by "ESound Support", you'll see that it's good for Linux only, and that ESD must be running. Luckily, you already started ESD with this command.

The following is the code to "pick off" data from /dev/dsp and convert it to .ogg:

esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o $1 -

The esdmon program "picks off" the digital audio from /dev/dsp and outputs it to stdout. Via a pipe, that output is sent to the sox program, where the sampling rate is adjusted to match that set in the esd program, the filetype is set to raw because the output of esdmon has no header, -c is set to 2, meaning 2 channels (use 1 if the original .ra was monaural or if you want to combine channels to save disk space), specify sampling in 16 bit words, signed linear sample encoding, output .wav format, and adjust the volume to best match native .ogg files.

Now that .wav output is produced, the final step is to convert the .wav to .ogg. This is done by the oggenc program, with options to resample at the original .ra file's sample rate, output to a specific file, and accept the incoming input as raw data.

The following table shows the meaning of the options for the sox and oggenc portions of the command:

sox arguments and options oggenc arguments and options

sox	Sound translation program
-r 11025	Resample at 11025
-v2	Crank up the volume to level 2
-c 2	2 channels (stereo)
-t raw	Incoming data is raw (no header)
-w	Incoming data samples are 16 bit words
-s	Incoming data samples encoded as signed linear (2's complement)
-	Incoming data read from stdin
-t wav	Outgoing data is `.wav` format
-	Outgoing data is sent to stdout

oggenc	The ogg encoder
-r	Incoming data is raw
--resample 11025	Resample at 11025
-o $1	Output file is arg1
-	Get input from stdin

Symptoms and Solutions

Although at this time (7/20/2004) the realplay program can be obtained without payment, it is a proprietary black box depending on many unknown (and unknowable because you don't have the source) variables. Therefore, it can act intermittent with respect to computer, OS installation, reboot, time, and who knows what else. These unknowns make troubleshooting very difficult.

Perhaps the best you can hope for is to get it up long enough to convert your .ra files. It might even require installing a different distro, possibly an older distro, on a separate box.

This symptom and solution article attempts to guide you through solutions to the most common problems. Keep in mind that these solutions might not always work, and even more perplexingly, the listed error modes might not produce the listed error conditions. Do your best, nurse realplaythrough the conversions, and in the future try not to record or obtain anything in .ra format.

Realplayer doesn't run, window mostly black

This typically occurs when the esd program is running, but realplay is set to use native sound drivers. The solution is to kill the esd program, wait for a while, then try again. If you look in the /etc/esd.conf file you'll see a delay called spawn_wait_ms=300 and another delay -as 2. These are two different delay intervals that might, or might not need to be exceeded before the killing of the esd program takes effect.

Once you can get realplay running correctly, you can manipulate its menu to switch it to use ESound:

View->Preferences->Performance_tab->ESound_Support

From that point, exit realplay and in a different command prompt run esd:

esd  -r 11025 -d /dev/dsp

Cannot open the audio device

The complete error message might look something like this: "Cannot open the audio device. Another application may be using it." First, check to make sure no other sound programs are running, and shut them down if they are. If that doesn't help, the esd program probably isn't running or isn't running right. Exit the realplay program and run esd. Assuming the .ra program was recorded at 11025 samples per second, it would look something like this:

esd  -r 11025 -d /dev/dsp

Then run realplay. It should now play its opening welcome message, and should be able to open and play other .ra programs.

Inability to play more than one realaudio clip

4Occasionally you'll get the "Cannot open the audio device. Another application may be using it." message when trying to play a second sound clip. Under these circumstances, if you look at the window running the esd command, you'll probably find that esd terminates after the completion of a realaudio clip. The way I've found to make esd more durable is to modify /etc/esd.conf. The original probably looks something like this:

[esd]
auto_spawn=1
spawn_options=-terminate -nobeeps -as 2
spawn_wait_ms=300

BACK UP YOUR EXISTING /etc/esd.conf!!! Then modify /etc/esd.conf. Try eliminating the -terminate and then add the -d and-r to it:

[esd]
auto_spawn=1
spawn_options=-nobeeps -as 2 -r 11025 -d /dev/dsp
spawn_wait_ms=300

Now you can run esd with this simple command:

esd

Be aware that the preceding is a coathanger solution, and like all coathanger solutions that "fix" the symptom rather than eliminating the cause, it will cause side effects on all other programs using ESD. Therefore, once you've converted your .ra files, be sure to restore /etc/esd.conf from backup. If you desire, you can save the tweaked file as /etc/esd.conf.ra2ogg.

Distortion: Sound displeasing, screachy, overly treble, oscillatory, ringy, scratchy, noisy, artifacts

Bad sound, especially distortion of high frequencies, is typically caused by a too-high sample rate, an off-divisor sample rate, or even a too low sample rate. The first step is to try the exact sample rate of the .ra clip. The realplay window gives an approximation of the sample rate. The exact rate can be found by playing the clip, and while the clip is playing, do this: View->Statistics.

Try running esd with the -r value as the exact rate reported by View->Statistics. If that sounds bad, try finding a simple fractional divisor near that amount. For instance, 44100/(25/9) is 15876. If such attempts still sound bad, trial and error may be necessary. Start with 11025, and move up and down to see the improvement or degradation. Remember, you're looking for something not too harsh and not too muffled. Also, remember that you'll likely have some distortion no matter what you do. RealAudio was meant for efficient web streaming, not for great fidelity, and you can't completely recover the information from its highly compressed format.

Remember also that grossly low rates introduce distortion, often in the form of echos and other artifacts. Also, if you go too low, esd refuses to run at that rate and instead defaults to 44100 with 8 bit samples. 8 bit samples are guaranteed to give horrible sound.

The following table lists some sample rates:

Rate	Divisor	Fidelity	Comments
44100	44100/1	Full fidelity	CD quality
22050	44100/2	half rate	You'd need good ears to discern the decreased quality
16000	44100/2.75625		Probably won't convert well at this rate
15876	44100/(25/9)	custom divisor
14700	44100/3	third rate	Decreased highs, but still enjoyable
11025	44100/4	fourth rate	Lowest sample rate enjoyable as music
8820	44100/5	fifth rate	Good, understandable speech
4410	44100/10	tenth rate	Muffled speech, but reasonable understandable. A little worse than telephone quality. Consonants such as S and F cannot be discerned except by context.

Another source of distortion is a too high volume multiplier in the sox program. This distortion sounds more like ordinary clipping (volume too high on an amplifier) rather than the high frequency phenomenon caused by a too high sample rate. You can test by reducing the volume multiplier to 0.5 and boosting the speaker volume to a hearable level. If the distortion goes away, the problem was a too high volume multiplier. If not, it was something else.

Song plays too fast or too slow

Check all your sox and oggenc options. Try them identical to those in this document, and see if the problem persists.

Mass Conversion

What if you have a directory full of .ra files all recorded at similar volume and at identical sample rates? As far as I know, realplay itself doesn't have a mass conversion facility. Theoretically, you could convert the whole playlist to a single .ogg, and then use an audio editor to break it into individual songs. Do it if practical.

I found the following to be effective, although requiring intensive operator intervention. First, create the following esdmon.sh:

esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o $1 -

Now create the following do1.sh, which converts a single song contained in /mnt/songs:

FN=$1
gnome-terminal -e "./esdmon.sh $FN.ogg -" &
~/RealPlayer8/realplay /mnt/songs/$FN.ra

In the preceding, note that arg1 is merely the filename without extension, and without directory path. Thus, to convert /mnt/songs/mysong.ra to ./mysong.ogg, execute the following command:

./esdmon.sh mysong

Look closely at the do1.sh script. Notice that for each song, it runs esdmon.sh in a terminal in the background, and then runs realplay, on the appropriate .ra song, in the foreground. Thus, the conversion script is guaranteed to be running before the song starts in realplay, thus preventing elimination of the front of the song. However, the only way to stop esdmon.sh is to Ctrl+C the terminal BEFORE closing realplay. If you had closed realplay before terminating the terminal, you'd have gotten multiple terminals, and who knows what that would have done to your audio.

Finally, create a script to repeatedly run do1.sh on every song in /mnt/songs:

./do1.sh mysong
./do1.sh yoursong
./do1.sh freds_song

Run the script, and notice a terminal running esdmon.sh and a realplay window are immediately started, with music playing and the terminal counting time and marking movement with a spinning stick prompt.

For each song, do the following:

Wait for the spinning stick to stop
Ctrl+C the terminal to terminate esdmon.sh
Close the realplay program. A terminal and realplay window for the next song automatically runs.

Immediately Converting `.ram` Files

I haven't tried it, but I believe that this method can also be used to create the pure streaming version of RealAudio files, .ram files. You'd simply run esd with the proper options, run the esdmon | sox | oggenc command chain discussed previously in this document, and then play the piece. As long as your realplay is set up for the ESound driver, it should work.

However, keep in mind that as time goes on, the likelihood of realplay's continued Linux availability and Linux functionality decreases. It's best to try to distance yourself from the .ra format entirely, and concentrate on .ogg, or else on .mpg, which you can then convert to .ogg.

Summary

The .ra format is a great way to ship music over a modem connection to be played realtime on the other end, but it's not particularly high fidelity. Furthermore, it's primary playback mechanism is The Real folks' realplay, and there's no guarantee of continued Linux availability and function of realplay. If realplay becomes unusable on Linux and nothing else takes its place, your .ra files become unplayable AND unconvertable. Perhaps they could be converted on a Windows box. Perhaps!

In other words, it's time to convert any .ra files you own to another format. But what format? .mp3 is patent encumbered, meaning at any time huge fees could be charged for it. Remember how Unisys, the owner of the lzw compression used in .zip files, tried to impose large fees on webmasters displaying.gif files? This could happen with .mp3 files.

The .wav and many other formats are too big. The Windows Media Player format is proprietary making it not much better than .ra.

Then there's .ogg. It's patent free, so you can use it with confidence now and in the future. The only way to encumber .ogg is for the government to pass specific laws banning it as part of a "manditory digital rights management" law. If that happens, our society, especially Linux users, face much more daunting concerns that loss of songs. Such a law would make Linux itself illegal.

So the question is "how to convert from .ra to .ogg? Here are the basics:

Find out the sample frequency of the original .ra file(s)

Throughout this document we assume 11025

Set realplay to output ESound

View->Preferences->Performance_tab->ESound_Support

Run ESD

```
esd  -r 11025 -d /dev/dsp
```

Pipe and translate /dev/dsp to out.ogg:

esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o out.ogg -

Immediately play the song

~/RealPlayer8/realplay mysong.ra

When the song stops playing and the spinning stick stops spinning

Ctrl+C out of the esdmon command

Close realplay

The course of conversion seldom runs smooth. If you have problems, look in the Symptoms and Solutions section. If you want to know a little more about the underlying technology, take a look at the Principles of .ra and .ogg section. Mass conversion and immediate stream conversion are possible.