Saturday 28 July 2018

A Local Deb Package Mirror

     (( with Devuan xinetd and approx ))


There are a number of ways to build a shareable "mirroring cache" of packages on a non public Network. This saves bandwidth, time, and possibly enables disconnected use with some small tweaks.
This post is a shorter description of one possible process. There is a longer even more verbose version one post later.

e.g. These proxies all scale to small enterprise level at least.
I have used them all successfully at some time or other.

  * squid-deb-proxy - Squid proxy configuration to optimise package downloads
  * apt-cacher-ng - caching proxy server for software repositories
  * approx - caching proxy server for Debian archive files


The squid approach (? probably ?) supports multiple connections better than the last two. Though thats just a guess, for more industrial uses its worth at least a look. It works OK on a home LAN but not on any machine you reboot a lot.
Waiting for squid to shutdown can get "old" quite quickly.


This Walk through documents setting up a HTTP based proxy using
*approx_ and xinetd_*.

Features of this approach include:

  - Reliable and _low_ maintenance
  - Small and light enough for home use
  - efficient enough for enterprise level deployment.
    Should scale up to 100's of LAN clients including virtual systems.
    (depends on hardware and bandwidth)

  - Tunable access by network space or whitelist
    and optionally time of day. (xinetd)
    (No need to worry your iptables or other filtering.)
  - Tunable system loads and concurrency.  (xinetd)
   
  - A simply structured *pool based mirror*  under /var/cache/approx  (default).
  - Readily and cleanly supports other Debian type systems
    and 3rd party repositories
 
  - Low standby resource demand (xinetd)
  - Fast start up and shut down. (xinetd)

  - Logs flexibly. To file or syslog. ( approx and / or xinetd )

The downsides are:
  - editing the clients sources.list . (though there are options around that even)
  - setting up xinetd the first time may be an issue. 
  - this is not a simple apt-get install  and walk away.
_You_ will have to create at least one new file and edit all your clients sources.lists.

However its really not difficult take your time and test as you go.

The Shortest Version

root@yours~# apt-get install xinetd approx

The Short Version

Adjust values to suit network(s) and remote repositories.

root@yours~# apt-get install xinetd approx

 1. edit the servers
/etc/approx/approx.conf
::
    devuan http://deb.devuan.org/merged



 2. edit a clients sources.list. e.g.
/etc/apt/sources.list.d/01-stable.list
    (? disable the rest while you test ?)
    *The client can be the same machine as the server.*
   
 ::
    deb http://10.1.1.10:9999/devuan stable  main contrib non-free

 3. create xinetd approx service file at
/etc/xinetd.d/approx
   
 ::
    # /etc/xinetd.d/approx
    service approx
    {
    id           = approx
    disable      = no
    bind         = 10.1.1.10   
    port         = 9999
    only_from    = 127.0.0.1/8 10.1.1.1/8
    protocol     = tcp
    socket_type  = stream
    wait         = no
    user         = approx
    server       = /usr/sbin/approx
    }

4. Restart xinetd

::
   root@yours~#  /etc/init.d/xinetd  restart

   (or)

   root@yours~#  service xinetd restart

check xinetd has enabled your new service ::

    you@yours~$  netstat -tan | grep 9999
   
    you@yours~$  nc 10.1.1.10 9999

        "get foo"

        HTTP/1.1 400 Bad Request
        Server: approx/5.10 Ocamlnet/4.1.2


(yes that *nc get*  is deliberately broken)

    you@yours~$  CTL c
  
to close the "conversation".

5. check default logging (and any errors like the one above)

::
  you@yours~$  tail -f /var/log/daemonlog

something like: *Nethttpd: Bad request line*

6. simple test from the configured client

::
   root@yours~# apt-get update

see:    /var/cache/approx/

next:
The Long Version includes more options and some alternatives.

References:

Devuan
Xinetd
Approx

No comments:

Post a Comment

  A Local Devuan Package Mirror  (( with Xinetd and approx ))   Verbose Version A shorter simpler version is also available (one ...