2.1
What RAR and PAR files are for?
2.2 Naming conventions
2.3 Downloading, recovering, combining
and unpacking them
2.4 Gathering RAR and PAR files
from multiple servers
2.5 Posting multipart files with
parity sets
2.6 PAR2 file format
1.
yEnc FAQ
1.1
Why is there a need for encoding of binary files at all?
Originally,
the Usenet was invented for discussions, i.e., exchange by textual messages.
So, Usenet software and protocols were developed with this limitation
in mind: all messages are textual.
When it was
realised that it would be good to be able to post binary files as well,
it presented a problem. Binary files, e.g. pictures, contain all sorts
of byte codes, not only textual. Posting them meant violating the "text-only"
limitation that had been built into Usenet software and protocols. So,
the software would not accept binary files or would not work properly
with them.
This problem
found a simple solution: let's encode binary files in such a way that
they contain only acceptable byte codes, and then we can post them.
This is what encoding is for. Normally, encoding and decoding is performed
transparently by your mail and news software and you don't have to know
anything about it.
1.2
How does encoding work?
All encoding
methods (uuencode, xxencode, base64 and binhex), except yEnc, use the
same basic principle: they replace every 3 bytes of the original binary
file with 4 bytes from an encoding table that contains only textual
codes. It is a bit more complex than that, but this is the idea.
When you download
an encoded file from the Usenet, your decoder performs the reverse transformation
and you end up with the original binary file.
Please note
that the size of the encoded file is at least 33% larger than the size
of the original binary file.
1.3
What is yEnc and what is different about it?
yEnc is
based on the recognition that not all "non-textual" byte codes
are "bad" for the Usenet, but only a few of them. The rest
can be posted and processed by Usenet software without any problems.
yEnc encodes
binary files in two steps: first, it changes each byte by a constant
in order to minimise the amount of "bad" bytes in the file.
Second, it replaces each "bad" byte with a unique sequence
of two bytes. Again, it is a little bit more complex than that, but
not much.
As far as we
know, this process increases the size of the encoded file only by 2-3%
on average.
1.4
What are advantages of yEnc?
The advantage is obvious: about 30% of encoded file
size savings. This means that posting and downloading binary files take
about 30% less bandwidth and your time. Also, please keep in mind that
binary groups consume most of the Usenet resources (news feed traffic
and server storage space). Here is where yEnc brings savings, too.
1.5
Is yEnc safe?
We don't
see why yEnc is less safe than any other encoding method.
Generally, the
Usenet is not very safe at all. Anyone can post infected software or
just a bomb that will wipe out your hard drive. There are claims that
playing music and videos downloaded from the Internet can infect your
computer.
If you are concerned
with your privacy and security of your computer (most people are), there
are a few things that you should do:
- Do
use an anti-virus. There are plenty available.
- Do
use a firewall. Firewalls protect you from hackers attacks. Norton
Personal Firewall is a good choice. We tried two other
quite popular firewalls, but they were causing problems.
- Do
check your computer for spyware regularly and remove it. A good free
tool is Ad-Aware
(recommended).
- Do
not start executables and .bat files downloaded from the Usenet
unless you are absolutely sure that it is safe.
1.6.
I have a very old slow PC. It works all right for me now. Will it cope
with yEnc?
You should not
notice any difference.
1.7
My newsreader does not support yEnc. What can I do to download yEnc
encoded files (pictures, music…)?
There
are a few options that you have:
- Ask your
newsreader developers whether they are going to implement yEnc decoding.
Chances are, they are going, but you would have to wait.
- You can try
to download messages containing yEnc encoded attachments and save
them as files on disk, then run a yEnc decoder on them, available
from here.
You may succeed, but it is not guaranteed, because in the process
of working with a yEnc encoded message your newsreader may correct
"inappropriate" bytes and thus, corrupt the encoded file.
- Find another
newsreader that does support yEnc. There are plenty by now. Most people
reading this are Outlook users. Outlook is a very good tool, but not
a binary newsreader. If you are interested in yEnc, you must be working
with binaries. Get a newsreader designed for that.
- There
are a couple of applications that you can use as yEnc proxies.
A yEnc proxy is a program that sits between a news server and users
newsreaders and replaces yEnc encoded attachments with the same attachments
encoded using some other method. Thus, newsreaders that do not support
yEnc, can decode attachments that were originally encoded using yEnc
if they receive them via an yEnc proxy. You can run such a program
on your own computer. First one of these applications is Ozway.
The other one is yProxy.
They are both free, but slightly different.
Ozway was
designed and built to work efficiently primarily in multiuser mode.
It is used by ISPs and commercial news providers. Ozway is multiplatform
(Windows, FreeBSD and Linux), fast and does not have a practical
limit of number of simultaneous connections. However, it is also
very small (smaller than yProxy) and can be used by individuals.
yProxy,
though can be shared, is an individual application. It has a limit
of 10 simultaneous connections.
Both Ozway
and yProxy are limited to be used with one news server, but you
can install and run several copies, to access multiple news servers
simultaneously. yProxy has a GUI interface, Ozway relies on text
configuration files. yProxy is simpler, but Ozway seems to be more
reliable.
1.8
I downloaded a set of files and ended up with files with extensions
".ntx". What can I do about them?
Some posters
do very irrational (to put it politely) thing: they first encode files
manually with yEnc encoder, than post these files. In the process of
posting, their posting software encodes these files again using some
other method, for example, uuencode.
Then, when you
download such files, your newsreader recognises that they are uuencoded
and decodes them. But, it is not aware that that was only the second
level of encoding. So, what you have now are yEnc encoded files. To
get the original files, run yEnc decoder on them. You can download it
from here.
1.9
I am a developer. Where can I get yEnc specs and sample code?
From the
yEnc Web site. There is also an FAQ for more advanced users
on this site.
1.10
How about posting in yEnc format?
You can try
free Ozum autoposter. Starting with version 2.85, Ozum supports
automatic posting of collections of files in UUencoding and yEnc formats,
depending on your choice.
1.11
Suddenly, my newsreader can't decode yEnc articles anymore. What's happening?
This topic has
been discussed multiple times in the newsgroups. Try disabling your
firewall. If this helps, you know what to blame.
2.
RAR and PAR Files FAQ
2.1
What RAR and PAR files are for?
RAR
(.rar) files are created by WinRAR archiver.
This format makes it easy to create multipart volumes and therefore
it is easy to split large files into parts using WinRAR. This is the
preferred format for posting of compressed files on the Usenet. Posting
in other formats, such as zip and ice does happen, but is strongly discouraged.
Generally,
posting compressed files in the News does not seem to be a very good
idea, especially posting images. If they are posted in an archive, you
have to download the entire archive even if you want to have only a
few of them.
Size reduction
benefits achieved by compressing images, movies and mp3 sound files
are very insignificant because contents of these files are already compressed
using medium-specific compression techniques, and it is hard to compress
them more without loss of quality.
It seems that
the only thing that justifies using rar format for posting, is the ability
to create multipart volumes and error recovery files.
Error recovery
files (parity sets, par) are redandant additions that often are
posted with multipart files, such as rar archives or files created by
MasterSplitter. When available, parity sets can be used to recover
any missing part of the file. As many parts can be recovered as
there are parity sets available. Parity sets are good for error recovery
and nothing else. You don't need to download parity sets or even
think about them if you have all parts of the multipart file available.
2.2
Naming conventions
There
are two forms of file names generated by WinRAR. The first one
is <name>.part##.rar, where '##' is part number, starting with
01. Example 1:
Moody Blues
Days of the Future Passed MFSL.part01.rar
Moody Blues Days of the Future Passed MFSL.part02.rar
Moody Blues Days of the Future Passed MFSL.part03.rar
Moody Blues Days of the Future Passed MFSL.part04.rar
...
The second
form is <name>.r##, where '##' is either 'ar' or part number,
starting with 00. Example 2:
Beatles
- Rubber Soul.rar
Beatles - Rubber Soul.r00
Beatles - Rubber Soul.r01
Beatles - Rubber Soul.r02
Beatles - Rubber Soul.r03
...
Parity sets,
or error recovery files, can be posted with RAR files named in any
of these two ways. Parity sets are normally named <name>.P## where
'##' is set number, starting with 01. A file with name <name>.par
is normally included in the post and contains information used for parts
checking and recovery. It is sort of a header/summary/info file.
Example 1 plus
parity sets:
Moody
Blues Days of the Future Passed MFSL.part01.rar
Moody Blues Days of the Future Passed MFSL.part02.rar
Moody Blues Days of the Future Passed MFSL.part03.rar
Moody Blues Days of the Future Passed MFSL.part04.rar
...
Moody Blues Days of the Future Passed MFSL.par
Moody Blues Days of the Future Passed MFSL.P01
Moody Blues Days of the Future Passed MFSL.P02
Moody Blues Days of the Future Passed MFSL.P03
...
Example 2 plus
parity sets:
Beatles - Rubber
Soul.rar
Beatles - Rubber Soul.r00
Beatles - Rubber Soul.r01
Beatles - Rubber Soul.r02
Beatles - Rubber Soul.r03
...
Beatles - Rubber Soul.par
Beatles - Rubber Soul.P01
Beatles - Rubber Soul.P02
Beatles - Rubber Soul.P03
Beatles - Rubber Soul.P04
...
Please note
that parity sets can be created for any multipart set, not necesseraly
a rar file. For example, they can be created for and posted with files
split by MasterSplitter:
my_loved_kittens.mpeg.001
my_loved_kittens.mpeg.002
my_loved_kittens.mpeg.003
my_loved_kittens.mpeg.004
my_loved_kittens.mpeg.005
...
my_loved_kittens.mpeg.par
my_loved_kittens.mpeg.P01
my_loved_kittens.mpeg.P02
my_loved_kittens.mpeg.P03
...
2.3 Downloading,
recovering, combining and unpacking them
If
using Ozum
3.0 or later, double click on any article carrying any part
of a multipart file. Ozum will recognize the name format automatically,
will search the list of articles for the .par (info) file and other
parts, check if they are complete, check if parity sets are available
present this information to you and then download, recover (if needed)
assemble and unpack the file automatically.
Read
further only if using other newsreaders, such as Outlook Express etc.
You
will need some extra software. You can download WinRAR (shareware) from
this site:
http://www.rarlab.com/download.htm
You
may need SmartPar to recover missing file parts using parity sets SmartPar
is free. You can download SmartPar and a lot of other par related tools
from this site:
http://parchive.sourceforge.net/
Associate
.par extensions with SmartPar and .rar extensions with WinRAR.
- Download
all parts to a single directory. By "parts", we mean whole
files with names as described in 3.2 above. Such files usially are
split again when posted. The way to get them together, depends on
your newsreader.
- If some parts
are missing, download and save to the same directory as many parity
set files as there are parts missing. You can use any parity set file
to replace any part.
- Download
and save to the same directory .par file included in the post.
- Open the
directory in Windows Explorer..
- Double click
on the .par file. SmartPar should start and check the file(s) and
recover missing part(s) if any.
- Open .rar
file in WinRAR. You can unpack it now.
2.4
Gathering RAR and PAR files from multiple servers
You can
gather your files from multiple servers using Ozum. To get a file together,
do this:
- Download
all parts to a single directory. By "parts", we mean whole
files with names as described in 3.14.2 above. Such files usially
are split again when posted. Ozum will combine them automatically
for you
- If some parts
are missing, download and save to the same directory as many parity
set files as there are parts missing. You can use any parity set file
to replace any part.
- Download
and save to the same directory .par file included in the post.
- Open the
directory in the directory tree on the left. You will see a list of
files in the directory.
- Double click
on the .par file. If all the necessary files are there, Ozum will
automatically recover missing parts using parity sets, if needed,
then assemble the original file and unpack it to the current directory.
If you are using
other newsreaders, do steps 1-3. To do steps 4 and 5, you need extra
software. See 3.3.
2.5
Posting multipart files with parity sets
Use
WinRAR to create multipart archives. WinRAR is shareware. To the best
of our knowledge, there are no other tools to create archives in rar
format because it is patented. You can download WinRAR from this site:
http://www.rarlab.com/download.htm
Use
SmartPar to create parity sets (error recovery files) for your multipart
archive. SmartPar is free. You can download SmartPar and a lot of other
par related tools from this site:
http://parchive.sourceforge.net/
When
done, post the resulting files, including the created .par file.
2.6
PAR2 file format
This
is also a parity sets format. You can recognise files in this format
by .par2 extensions and '+' in extensions of some filenames.
If you come accross this format, download the files and parity sets
(if needed), and use free QuickPar utility to process them. You can
find QuickPar here:
http://parchive.sourceforge.net/
