Archive for the ‘linux’ Category

Incremental backups with duplicity, plus Nagios monitoring

septembre 20, 2017

The idea is to setup daily encrypted incremental backups.

This scheme involves 3 machines: the main server to be backed up (www.test.com), the backup server (backup.test.com) and a nagios server (nagios.test.com) for monitoring the backups. There can be as many servers and folders to be backed of as needed of course.

All the following was done on Linux Ubuntu machines. There’s not much details, use at your own risk.

 

Initial setup

On Your system administrator machine

  1. Prepare the GPG key to be used (adapt as needed)
    $ gpg --gen-key
    $ gpg --export D68E26B0 > sysadmin.public.gpg.asc
    
  2. Keep the private key safe, you’ll need it to restore backups.

 

On servers to be backed up

  1. Install/Upgrade duplicity
    $ sudo add-apt-repository ppa:duplicity-team/ppa
    $ sudo apt-get update
    $ apt-get install duplicity
    
  2. Copy the public gpg key, import it as root, and trust it (ultimate):
    # gpg --import sysadmin.public.gpg.asc
    # gpg --edit-key D68E26B0
    gpg> trust
    
  3. Test backup
    # LANG=en_US.UTF-8 duplicity --encrypt-key D68E26B0 /etc sftp://user@backup.test.com/duplicity/www.test.com/etc
    # LANG=en_US.UTF-8 duplicity collection-status sftp://user@backup.test.com/duplicity/test/etc
    
  4. Setup backup cron
    @daily LANG=en_US duplicity --encrypt-key D68E26B0 /etc sftp://user@backup.test.com/duplicity/www.test.com/etc
    

 

Nagios monitoring

The idea is to setup a probe using the duplicity’s collection-status command to test the backup freshness.

  1. Create a SSH key for nagios user (ssh-keygen). Since nagios user has no shell associated it must be created for another user and then copied to nagios’ home .ssh/ The .ssh/known_hosts file must also be setup to contain the backup machine key certificate.
  2. Setup Nagios on nagios.test.com to monitor your main server using the simple script below. This implies you are doing a daily backup, adapt as needed.
    • /usr/local/nagios/libexec/check_duplicity_backup
      #!/bin/bash                                                                                                                                                                      
      #Verifying if the backup plan is working or not                                                                                                                                  
      export LANG=en_US
      BACKUP_BASE_DIR=sftp://user@backup.test.com/duplicity/
      Host=$1
      Folder=$2
      TMP=$(mktemp)
      
      duplicity collection-status $BACKUP_BASE_DIR$Host/$Folder > $TMP 2> /dev/null
      exitcode=$?
      
      TODAY=$(LANG=en_US date +"%a %b %d")
      YESTERDAY=$(LANG=en_US date +"%a %b %d" -d yesterday)
      Latest=$(egrep "^Chain end time:" $TMP)
      
      rm -f $TMP
      if [[ $1 == "" ]]; then echo "Critical - Configuration Broken"; exit 2; fi
      if [[ $exitcode != 0 ]]; then echo "Critical - Check command failed"; exit 2; fi
      if [[ $Latest == "" ]]; then echo "Critical - No backup found at $BACKUP_BASE_DIR$Host/$Folder"; exit 2; fi
      if [[ $Latest == *$TODAY* ]]
      then
          echo "OK - $Latest" 
          exit 0
      elif [[ $Latest == *$YESTERDAY* ]]
      then
          echo "Warning - $Latest" 
          exit 1
      else
          echo "Critical - $Latest" 
          exit 2
      fi
      
    • commands.cfg
      define command{
              command_name    check_duplicity_backup
              command_line    $USER1$/check_duplicity_backup $ARG1$ $ARG2$ 
              }
      
    • http://www.test.com.cfg
      define service{
              use                             generic-service         ; Name of service template to use
              host_name                       www.test.com
              service_description             Backup /etc/
              check_command                   check_duplicity_backup!www.test.com!etc
              first_notification_delay 0
              }
      

 

Publicités

Installer Mandriva sur une dedibox

mai 11, 2009

Ayant une dedibox originale (processeur Via C7) qui était devenue poussive, je me suis dit qu’il était temps de migrer vers une configuration un peu plus costaud.

« Buggar!! » comme dirait un ami à moi: plus de Mandriva sur les nouvelles Dedibox!

Me voilà donc parti pour plusieurs jours de galère pour essayer de migrer mon ancienne configuration. Ayant réussi à démarrer sur un kernel Mandriva, voici la méthode que j’ai utilisée. Un grand merci à Antoine Ginies pour son aide.

Attention: réservé aux utilisateurs Linux avertis! Utilisez cette procédure à vos risques et périls!

  1. Installer Fedora sur la nouvelle Dedibox
  2. Passer la dedibox en système de secours grâce à la console de gestion noter l’identifiant (ID) le mot de passe et l’adresse (IP)
  3. Utiliser le script suivant pour monter le disque dur et passer en chroot
    ssh ID@IP
    sudo mount /dev/sda1 /mnt/
    cd /mnt
    sudo mount -o bind /dev /mnt/dev
    sudo mount -t proc none /mnt/proc
    sudo chroot /mnt /bin/bash
    su - root
  4. À partir d’une machine sous Mandriva transférer le système vers la nouvelle Dedibox:
    tar cf - bin/  boot/   etc/  home/  initrd/  lib/   media/  mnt/  opt/   root/  sbin/   tmp/  usr/  var/ | ssh ID@IP "dd of=/mnt/mandriva.tar"
  5. Supprimer ensuite la Fedora pour la remplacer par la Mandriva (depuis le chroot de la nouvelle dedibox):
    rm -rf selinux/ bin/  boot/   etc/  home/  initrd/  lib/   media/  mnt/  opt/   root/  sbin/   tmp/  usr/  var/
    tar xf mandriva.tar
  6. Voilà la partie la plus délicate: il s’agit de regénérer un initrd qui fonctionne sur la nouvelle dedibox. J’ai utilisé cette commande, mais cela peut varier selon le noyau que vous utilisez:
    mkinitrd --with=ext3 /boot/initrd-2.6.29.2-tmb-server-3mdv.img 2.6.29.2-tmb-server-3mdv
  7. S’assurer que /etc/lilo.conf est correctement configuré pour booter par défaut sur le initrd généré et le vmlinuz correspondant
  8. lancer « lilo »
  9. S’assurer que le module de la carte réseau est présent dans/etc/modprobe.conf:
    alias eth0 r8169
  10. s’assurer que l’interface réseau est en DHCP (drakconnect)
  11. repasser la dedibox en mode normal par la console de gestion et croiser les doigts

Bonne chance!

(more…)

Black & White Photography?

octobre 22, 2005

Modern cameras allow to take B&W pictures, but the pity is that you lose the opportunity to have the color one, unless you save as RAW as well.

Furthermore quality is not excellent in JPEG format.

However converting from color to B&W is not simple as this article explains.

Wildman found a very nice way to achieve excellent B&W pictures from raw ppm files:

convert inputColor.ppm -fx ‘(u.g*0.66+u.r*0.33+u.b*0.01)’ outputBW.tif

Sample B&W result

Ergonomy vs. features?

octobre 22, 2005

The endless dilema.

Though I am convinced it is possible to achieve highly complex actions through very little effort.

The problem rose again when I had to convince a newbie it was actually easy to use the photos out of my new dSLR camera. Generally the process to extract photos through a dedicated interface is long and tedious. However it is possible to achieve all that efficiently in a single click.

Look at the code below. Thanks to that I can:

  1. download photos from camera and sort them by date of day in folders
  2. remove possible duplicates if I did not erase camera images since last download
  3. convert RAW images to a usable format

All this in one single click!

#!/bin/bash

# Change this to where to store Photos
target=/home/multimedia/Images
camera= »USB PTP Class Camera »
date=$(date –iso-8601)
mkdir -p $target/$date/tmp
cd $target/$date/tmp
# Get all photos from camera
gphoto2 –quiet –camera $camera –port usb: -P
# Do not replace photos that were already uploaded this same day
cp -u $target/$date/tmp/* $target/$date
rm -rf $target/$date/tmp
cd $target/$date
# auto-rotate using exif info
exifautotran *.JPG
# If photos were not erased from camera since last upload, remove duplicates
for i in *.{JPG,CR2}; do
for f in $(find $target -name $i ! -samefile $target/$date/$i); do
if md5sum $f | sed -e « s, .*/, , » | md5sum –check; then
rm -f $i;
fi
done
done
# decode RAW images if not already done ?
# for i in *.CR2; do if [ ! -e $(basename $i .CR2).ppm ]; then dcraw -w $i; fi; done
# Show them!
gimv -d $target/$date