The Easy Way to Set Up a Local News Server

By Christophe Blaess (ccb@club-internet.fr).

Introduction

A few months ago, I decided to set up a local news server on my Linux Box, in order to read off-line the articles. Before then I read the news directly from the Usenet server of my Internet Provider, at the detriment of my phone bill...(Here, in France, even the local communications are rather expensive)

Before trying to install and set up one of the two classical news servers (Inn and CNews), I browsed a bit the Web, looking for a possible other product. I then discovered a very powerful small package "Leafnode" written by Arnt Gulbrandsen. It can be found in source form at ftp://ftp.troll.no/freebies/leafnode/ and the home page of this project is accessible at http://www.troll.no/freebies/leafnode.html.

This package (leafnode-0.9.tar.gz, 29 Kb) contains three little programs, very easy to install, and to use:

The main advantage of this system, is the transparent way it is inserted between the remote news server and the local newsreader. The remote Usenet server sees Leafnode exactly like a classical newsreader (like tin, trn, netscape,...) and the local news reader sees Leafnode just like a USENET server.

I have been happily using Leafnode for several months, and I would like to describe here the (very simple) steps to install and configure it. Then I will explain some hints to use it in a multi-users environment.

INSTALLING LEAFNODE

I've been using leafnode 0.8 for a few months, but I've recently upgraded to version 0.9. The process for installing it is the same as the previous version, but I've had a little problem, maybe due to my version of make, and I'll describe how I have fixed it.

You first need to download the source code from ftp://ftp.troll.no/freebies/leafnode/. You'll get leafnode-0.9.tar.gz.

Compiling the source code

Become root and do :

  # cd /usr/local/src
  # tar -xzf ~/leafnode-0.9.tar.gz
  # cd leafnode-0.9/
  # make

Everything must compile without any problem...

Installing leafnode

Make sure there is a "news" user and a "news" group on your system.

Then you can type:

  # make install

If "make install" complains when making directories in /var/spool/news/message.id/, you may have the same problem as I've had. To fix it, I've modified the Makefile, to insert a part of the leafnode-0.8 Makefile:

replace the lines (in the "install:" section)

  cd $(SPOOLDIR)/message.id
  for a in 0 1 2 3 4 5 6 7 8 9 ; do for b in 0 1 2 3 4 5 6 7 8 9 ; do \
     mkdir ${a}${b}0 ${a}${b}1 ${a}${b}2 ${a}${b}3 ${a}${b}4 ; \
     mkdir ${a}${b}5 ${a}${b}6 ${a}${b}7 ${a}${b}8 ${a}${b}9 ; done

by the line

  -mkdir -p $(SPOOLDIR)/message.id/{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}

Be sure that the first character on the line is a tabulation. Then try again:

  # make install

End of installation

At this point, Leafnode will have installed the following files on your system:

    /usr/local/sbin/fetch
    /usr/local/sbin/leafnode
    /usr/local/sbin/texpire
    /usr/local/man/fetch.8
    /usr/local/man/leafnode.8
    /usr/local/man/texpire.8
    /usr/lib/leafnode/config.example
    /var/spool/news/ ... and a lot of subdirectories ...

First you have to copy the file /usr/lib/leafnode/config.example to /usr/lib/leafnode/config and edit him, to put the name of your remote NNTP server in place of:

  server = news.hiof.no

Edit the file /etc/inetd.conf, and look for a line like:

  nntp   stream   tcp   nowait   root   /usr/sbin/tcpd   in.nntpd

then modify it like this:

  nntp   stream   tcp   nowait   news   /usr/sbin/tcpd   /usr/local/sbin/leafnode

Depending on your configuration, there may be no TCP wrapper installed, so you could have to remove the "/usr/sbin/tcpd" part of this line.

and do

   # killall -HUP inetd

First run

Connect to your Usenet provider, and, while being "root" or "news", run:

  $ fetch

You will have to wait for a moment, because Leafnode is asking the NNTP server the list of all the active groups. Once fetch ends, run a newsreader, as normal user, and ask him to contact the localhost. for example, with tin do:

  $ export NNTPSERVER=localhost
  $ tin -r

(With Netscape you have to put localhost in "Options/Mail and News/Servers/NNTP server")

You will get the list of all available newsgroups, then choose interesting ones, and read them. They will appear empty at this time. It's normal.

As root run again fetch. It will download all the content of the previously read newsgroups. The first downloading will take a while, but the next will obviously be very much quicker.

Your local USENET server is installed!

CONFIGURING LEAFNODE

Now that leafnode run on your Linux box, you can configure some details:

In the file /usr/lib/leafnode/config, there are two fields you can edit:

Refer to the comments in this file to choose the values (on my system I use expire=10 and maxcount=3000)

You must run Texpire from time to time, and a daily entry in the crontab seems to be the best choice:

  00  03  *  *  *  news  /usr/local/sbin/texpire

This line is for a system-wide crontab (generally /etc/crontab maintained by root), but you will have to remove the username "news" on user crontab (/var/spool/cron/news).

You will have to run periodically fetch. If you have a permanent link with your news server, there's an obvious solution: the crontab again, to run it once per hour for example.

  00   *  *  *  *  news  /usr/local/sbin/fetch

If you're using a non-permanent PPP connection, you can insert fetch at the end of the ip-up shell script (see Linux Gazette 7 "Setting up PPP's ip-up and ip-down scripts!").

Put the name "localhost" in /etc/nntpserver or set the environment variable (for example in /etc/profile):

	export NNTPSERVER=localhost =

USING LEAFNODE ON MULTI-USERS SITE.

There are problems to avoid, especially if there are other users on your system, or if the leafnode server is on a local network. First you must make sure of the validity of the headers in the outgoing posts, but you also need to limit the list of the accessible newsgroups. Don't forget that Fetch will download the whole content of a newsgroup if someone tries to read it. So, be very careful with newsgroups like alt.binaries.pictures...

Fortunately, we can use some little awk and shell scripts to verify and correct the outgoing posts, and to limit the local access to selected newsgroups.

Checking the outgoing posts

With some newsreaders, the "From:" field of the outgoing articles will be set to myname@my.station.on.my.lan and not myname@my.internet.provider.com. With some of them you can configure the "From:" and "Reply-to:" fields, while the others need you to recompile them.

This problem can be worse if you have a Linux box with several users. Some of them can have misconfigured newsreader (sometimes on purpose...) and it may be safer to check the headers of the outgoing articles before posting them.

Here's a small awk filter which allows a kind of masquerading of the "From:" line of an article. It will change the line "From: username@my.station.on.my.lan (user real name)" to a line "From: username@my.internet.provider.com (user real name)".

You may also ensure that username is correct (i.e. in a list of allowed users). The same Perl script will help us to determine the correct articles. Otherwise it will add a line "*** Wrong From field - This article must be deleted ***" to the message.

  #! /usr/bin/gawk -f
  #
  # /usr/local/sbin/change_article_from_domain
  #
  # awk script to change the domain name on the "From:"
  # line of outgoing articles.
  # If the username is not valid a message will be added
  # at the bottom o the file, allowing a 'grep' to delete
  # him.

  BEGIN {
    # replace with the correct domains
    local_domain="my.station.on.my.lan"
    real_domain ="my.internet.provider.com"
    # insert here the name of your users allowed to post articles
    # (may be just one)
    valid_usernames["user1"]
    valid_usernames["user2"]

    must_be_deleted=0
  }

  /^From:/ {
    gsub(local_domain, real_domain)
    username=substr($2,1,index ($2, "@")  - 1)
    if (! (username in valid_usernames)) {
      must_be_deleted=1
      # you can also add a system command
      # example : mail to newsmaster with
      # the username of the wrong article
    }
  }

  END {
    if (must_be_deleted != 0) {
      print "*** Wrong From field - This article must be deleted ***"
    }
  }

  {
    print
  }

This script can be useful if you have up to, say, ten users, otherwise you'll need to improve it in order to read the list of allowed users in an otherfile for example.

Now we will execute the above script on all the outgoing articles, sitting in /var/spool/news/out.going, then delete (or move to another directory) those with bad usernames.

  #! /bin/bash
  #
  # /usr/local/sbin/modify_outgoing_articles
  cd /var/spool/news/out.going
  for i in * ; do
    /usr/local/sbin/change_article_from_domain < $i >/tmp/modified_articles/$i
  done
  rm -f *
  mv /tmp/modified_articles/* .
  rm -f `grep -l "*** Wrong From field - This article must be deleted ***" *`

(The backquote is used to catch the result of grep) Don't forget to create a /tmp/modified_articles/ directory. This script cannot prevent the fake "From:" lines, when an authorized user is hidden behind an other correct username. This can not be easily done, and if you really don't trust your users, you'll have to use another Usenet package like Inn or Cnews.

Now all the outgoing articles will have a correct "From:" line.

Limiting the list of accessible news groups

The second important point to check out is the list of fetched newsgroups. If you haven't got a huge disk space, it would be better to avoid downloading alt.binaries groups or alt.2600. for example... The problem is that fetch will download the content of each newsgroup corresponding to a file in /var/spool/news/interesting.groups, for example /var/spool/news/interesting.groups/comp.os.linux.announce

A file in this directory is touched by leafnode every time a user tries to read the content of the group. Are you sure that none of your users will try to have a look at alt.binaries.pictures.erotica...? So there are two possible solutions:

  #! /bin/bash
  #
  # modify_interesting_groups

  cd /var/spool/news
  rm -f alt.*
  rm -f *windows*
  ...
  touch comp.os.linux.announce
  touch comp.windows.x.announce
  ...

you can also have a definite list of fetched newsgroups, for example in /var/spool/news/official.list, manually created:

  # mkdir /var/spool/news/official.list
  # cd /var/spool/news/official.list
  # touch comp.os.linux.announce
  # touch comp.os.linux.answer
  # touch comp.lang.c.moderated
  ...

And this list will be copied in the interesting.groups directory by a script before each execution of fetch.

  #! /bin/bash
  #
  # modify_interesting_groups

  cd /var/spool/news
  rm -f *
  copy /var/spool/news/official.list .

Now we have two scripts to insert before running fetch. For example in /etc/ppp/ip-up:

  #! /bin/bash
  #
  # /etc/ppp/ip-up
  ...
  /usr/local/sbin/modify_interesting_groups
  /usr/local/sbin/modify_outgoing_articles
  fetch

CONCLUSION

I think that Leafnode is a very interesting package for those (most of us) who are running Linux on a standalone box with intermitent connection to a Usenet server, or on a small local network with few users. It's powerful and much simpler to install and to configure than Inn or Cnews, designed for bigger sites. Moreover it does not require any maintenance.

Christophe Blaess (ccb@club-internet.fr).

[ TABLE OF CONTENTS ] [ FRONT PAGE ] Back Next