Aegir is a hosting system for managing Drupal sites.
Below are links to the project page, git repositories and community resources.
aegirproject.org
Bug tracker
Aegir Group
Git repos
Mailing lists
After much trial and grief, I've finally nailed how to have multiple SSL virtual hosts running on Apache on our Debian servers alongside non-SSL hosts.
In the 'default' apache config, put:
NameVirtualHost *:80
NameVirtualHost *:443
Then for each virtual host config that you ever create, the opening VirtualHost tag MUST include the relevant port as follows:
(for SSL site)
(for standard site)
Note that I have added a space between the tags due to Drupal attempting to format as HTML. Remove the spaces in your config file.
You must then have the ServerName specified inside each Virtual Host.
Somehow it didn't seem that easy at the start!
The following rewrite rule enables all requests to a site to go through via its www URL, theoretically improving search engine 'optimization' (if there is such a thing).
The important thing here is that any file or directory request following the url is kept during the redirect (I've seen poor rewrites on sites that just redirect from http://whatever.com to the front page and the user loses their path to the link they clicked on once already).
RewriteEngine On
RewriteCond %{HTTP_HOST} ^example\.com$
RewriteRule ^.*$ http://www.example.com%{REQUEST_URI} [R=permanent,L]
Experimenting with a bunch of VMs and CouchDB for
scalable, replicated filesystem purposes.
I like the way it stores any object as a 'document' in its database, doesn't depend on another
database backend like MogileFS seems to.
I'm slightly wary of the fact that the replication isn't built as part of the daemon's
responsibility automatically on the fly, by way of which you could control via the config ini
file. Instead replication is a manually executed process either via the web interface or via
curl commands that one could at least build into bash scripts... and maybe via the API i.e it
could be done in the app itself that is pushing files up into the database?
Not sure if that's a good feature or not yet.. also the fact that it's uni-directional
replication, making it not so easy to get a master-master HA solution happening.. you can
replicate both ways but what happens when conflict occurs (and they seem to)..
Watch this space!
#!/bin/bash
DATE=`date +%Y%m%d`
FILENAME=$1/$1_daily_$DATE.sql.bz2
mkdir -p $1
mysqldump --user=root --password=changeme --host=localhost $1 | bzip2 -c > ${FILENAME}
scp -i /home/web/.ssh/password_less_ssh_key $FILENAME <a href="mailto:user@remote.server">user@remote.server</a>:/data/backup/dbbackups/$1/
mysql -p DATABASE
(enter password)
SELECT * INTO OUTFILE "/tmp/table.txt" FROM TABLE;
#!/bin/bash
DATE=`date +%H`
FILENAME=$1/$1_hourly_$DATE.sql.bz2
mkdir -p $1
mysqldump --user=root --password=changeme --host=localhost $1 | bzip2 -c > ${FILENAME}
/etc/init.d/mysql stop
mysqld_safe --skip-grant-tables &
mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("newrootpassword") where user='root';
mysql> flush privileges;
mysql> quit
/etc/init.d/mysql stop
/etc/init.d/mysql start
mysql -u root -p
I seem to have to keep looking up notes whenever I need to set up a multi-master MySQL replication ring. I thought I'd put it all down in one place that I can find easily - on my own blog.
This is not a howto, it's just notes, I can't guarantee these are accurate or without faults.
1. Install MySQL server on Server A and B
apt-get install mysql-server
2. On Server A, grant replication privileges to a replication user
grant replication slave on *.* to 'replication'@'server_b' identified by 'slavepw';
3. MySQL config on Server A:
server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
binlog_do_db = testdb
binlog_ignore_db =mysql
relay-log=mysqld-relay-bin
Restart Server A MySQL
4) Edit Server B config:
server-id = 2
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
binlog_do_db = testdb
binlog_ignore_db =mysql
relay-log=mysqld-relay-bin
master-host = server_a
master-user = replication
master-password = slavepw
master-port = 3306
5) Restart MySQL on Server B, then:
start slave;
show slave status\G;
These should be both Yes, also check that the Master Host, binlog and positions all match (by comparing with 'show master status;' on Server A
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
At this point we've a master->slave relationship. It's time to make Server B a master of Server A as well.
6. Set the replication privileges on Server B to become a master of Server A
grant replication slave on *.* to 'replication'@'server_a' identified by 'slavepw';
7. On Server B, add the master info to my.cnf so it knows it's a slave of Server B
master-host = server_b
master-user = replication
master-password = slavepw
master-port = 3306
8. Restart Server B and then Server A
(not sure if it matters what order really)
9. On Server A:
start slave;
show slave status\G;
These should be both Yes, also check that the Master Host, binlog and positions all match (by comparing with 'show master status;' on Server B
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
10. start slave on Server B
(I guess, or was it already started when MySQL restarted? It should do it automatically, try to remember from 2008 what we did here to make sure it does)
Unless there's errors for either node connecting to each other, should be ok to create the database testdb on Primary or Secondary (the creation will replicate to the other node), and start creating tables/data on either node.
Backup script to find and backup all databases (postgres and mysql)
#!/bin/bash
today=$(date +%y%m%d)
# local dir where the backups go
myDir='/data/dbbackups'
# remote dir where we'll send them offsite
myRemoteDir='/data/backup/li35-166/dbbackups'
# this is for PostgreSQL. If you don't need it, you
# could leave it here, but remove the 'backup_pgsql'
# function call at the end of the script
function backup_pgsql {
for db in `su postgres -c "psql -U postgres -qAtc '\l'" | cut -f1 -d\|
| grep -v '^template[01]'`; do
mkdir -p $myDir/${db};
su postgres -c "pg_dump $db" | bzip2 > $myDir/${db}/${db}-$today.bz2;
done;
}
# MySQL backup function
function backup_mysql {
myFile='/etc/mysql/debian.cnf'
myUser=`cat $myFile | grep user | awk '{print $3}' | uniq`
myPass=`cat $myFile | grep password | awk '{print $3}' | uniq`
for db in `mysql -u$myUser -p$myPass -e "show databases" | cut -f2 -d\
| grep -v Database`; do
mkdir -p $myDir/${db};
mysqldump -u$myUser -p$myPass $db | bzip2 >
$myDir/${db}/${db}-$today.bz2;
done;
}
# here are the functions that actually get called.
# You might want to comment out the postgres one
backup_pgsql
backup_mysql
# rsync the contents of the local backup store
# to the remote machine. Since rsync is incremental
# this will only transfer today's dumps across,
# presuming that the remote end has all up until that
# point already
# obviously using ssh auth keys here
rsync -aHPq $myDir/ git.mig5.net:$myRemoteDir/
psql template1;
SELECT COUNT(*) FROM pg_stat_activity;
You can see max allowed connections with SHOW max_connections; - but this is just as easily read from the postgresql.conf in /etc/postgresql/(version).. etc
#!/bin/bash
DATE=`date +%Y%m%d`
FILENAME=$1/$1_daily_$DATE.sql
mkdir -p $1
pg_dump -cxO -f $FILENAME $1
gzip -f $FILENAME
echo $FILENAME
scp -i /home/user/.ssh/password_less_ssh_key $FILENAME.gz user@remote.backup:/data/backup/dbbackups/$1/
#!/bin/bash
DATE=`date +%H`
FILENAME=$1/$1_hourly_$DATE.sql
mkdir -p $1
pg_dump -cxO -f $FILENAME $1
gzip -f $FILENAME
echo $FILENAME
Tables:
SELECT pg_tables.tablename, pg_tables.schemaname, pg_size_pretty(pg_relation_size((pg_tables.schemaname::text || '.'::text) || pg_tables.tablename::text)) AS pg_size_pretty FROM pg_tables ORDER BY pg_relation_size((pg_tables.schemaname::text || '.'::text) || pg_tables.tablename::text) DESC;
Database entirely (I swear I had this sites statistics script somewhere that contained all this, including a MySQL version.. where did that go?):
select pg_database.datname, pg_size_pretty(pg_database_size(pg_database.datname)) AS size from pg_database WHERE pg_database.datname = 'foobar';
global {
usage-count no;
}
common {
syncer {
rate 100M;
}
}
resource nfs {
protocol C;
startup {
degr-wfc-timeout 120; # 2 minutes.
}
disk {
# on-io-error detach;
}
net {
cram-hmac-alg sha1;
shared-secret "removed";
}
on server-nfs-01 {
device /dev/drbd1;
disk /dev/mapper/ubuntu-SrvNfs;
address 10.0.1.91:7788;
meta-disk /dev/mapper/ubuntu-drbd[0];
}
on server-nfs-02
device /dev/drbd1;
disk /dev/mapper/ubuntu-SrvNfs;
address 10.0.1.92:7788;
meta-disk /dev/mapper/ubuntu-drbd[0];
}
}
# debug output
debugfile /var/log/ha-debug.log
# all other logs
logfile /var/log/ha-log.log
logfacility local0
keepalive 1
deadtime 10
warntime 3
initdead 20
bcast eth0
auto_failback off
# STONITH
#stonith_host * external/ssh 10.0.1.90 root nottherootpassword
node server-nfs-01
node server-nfs-02
server-nfs-01 charitynfs IPaddr::10.0.1.90/27/eth0 drbddisk::nfs Filesystem::/dev/drbd1::/srv/nfs::xfs
Before I forget... because it freaked the shit out of me and made me think I broke the postfixadmin database.
When you create a domain in Postfixadmin, that's all fine and dandy. But when you create Mailboxes or Alias in this domain, postfixadmin performs a nameserver lookup only at THIS point in time to check the domain is valid. The helpful response is:
"The EMAIL is not valid!" (for mailbox), or "The ALIAS is not valid!".
In fact, what it means is that the domain is not valid, so this is quite misleading. So don't panic!
I thought I broke things, but all that had happened was that for this particular subdomain, while I had created an MX record for it in the zone, I hadn't reloaded DNS yet, and so a nameserver lookup was failing.
You could wait for DNS propagation (still don't understand why it wasn't almost instant, mail server was also the primary NS), or a quick getaround is to change
$CONF['emailcheck_resolve_domain']='YES';
to
$CONF['emailcheck_resolve_domain']='NO';
In the config.inc.php of postfixadmin, and it'll skip the nameserver lookup.
Phew.
Be wary when you upgrade Debian Lenny server to 5.0.1 and you use Postfix & Dovecot with a PostgreSQL backend and PostgreSQL-8.3 gets upgraded and restarts itself. My Dovecot stopped being able to connect to the database to authenticate my login requests from Thunderbird all
of a sudden with the error 'you have enabled secure authentication and this server does not support it'.
I knew I hadn't changed any Dovecot settings on how I authenticate my IMAP session so assumed it was the PostgreSQL upgrade, though PostgreSQL was running fine.
This in the dovecot logs
dovecot: 2009-04-13 10:38:15 Error: auth(default): sql(<a href="mailto:example@some--address.com">example@some--address.com</a>),): Password query failed: FATAL: terminating connection due to administrator command
dovecot: 2009-04-13 10:38:15 Error: auth(default): io_loop_handle_remove: epoll_ctl(2, 9): Bad file descriptor
I restarted Dovecot, and Postfix just to be sure :) and all is back to normal. I'm assuming it was Dovecot holding a stale connection open to PostgreSQL, and not the MTA, as it wasn't a problem with delivering mail (though it might've posed a problem for Postfix too, but didn't let it hang around long enough to find out)
I wonder, I know that some packages like openssl and whatnot, are able to check what other services are running that depend on it when it's upgraded, and offer a postinst (I assume) ncurses notification telling the user that some services should be restarted. PostgreSQL should have a similar check to see if it's being used for a vhost mail setup.
Coz I always forget them!
http://www.fots.nl/index.php/ndr-and-smtp-reply-and-error-codes/
Login process creation
login_processes_count and login_max_processes_count setting control how new login processes are created. login_processes_count specifies the number of login processes that are tried to be kept listening for new connections. However when a lot of connections arrive at the same
time this number will increase automatically as described below.
To prevent fork-bombing Dovecot's login process creation works in a similiar way to Apache: Initially set "wanted listening process count" to same as login_processes_count and start the processes. Then check every second if we need to start up new processes to keep the listening
process count the same as the wanted count. If all the listening processes have been used, double the wanted count. If we haven't used all of them, decrease the wanted count by one (unless it goes below login_processes_count), but don't destroy already created processes.
login_max_processes_count specifies the maximum number of login processes there can exist at a time (listening, non-listening and SSL/TLS-proxying processes combined).
Didn't know you could do this.. both the ability to specify individuals rather than whole domains, and the retry: logic.
In the transport map
< user >@< destination.com > retry:4.0.0 Temporarily suspended
I was migrating my mail server to a new machine the other day and in the process, I sorted out my smtp/sasl logic.
I'm using virtual domains/mailboxes/aliases with Postfix, PostfixAdmin and Postgresql on Debian servers, works like a charm, but my sasl config was such that it wasn't using the same postfix database for password lookups and I was having to use saslpasswd2 to add accounts so that I could send mail using this server for SMTP as it was using its own sasldb or whatever.
Madness.
This time I've implemented Dovecot's SASL mechanism, it reduced double-up and means the authentication is done via the database in the same way that logging in to *retrieve* mail was being performed (as far as I understand it anyway. I'm not a mailserver guru).
All I changed was in /etc/dovecot/dovecot.conf
# It's possible to export the authentication interface to other programs:
socket listen {
client {
path = /var/spool/postfix/private/auth-client
mode = 0660 user = postfix group = postfix
}
}
And in /etc/postfix/main.cf
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth-client
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated,
reject_unauth_destination
Other than following the steps outlined in the INSTALL.txt file of Virtual Vacation’s directory in postfix admin, I had to do these to make it work properly as well:
Added
vacation_destination_recipient_limit = 1
to /etc/postfix/main.cf
At this point things work, so long as your recipient who's on holidays is not a member of any virtual aliases (ie he or she doesn't receive mail sent to generic addresses like info@ or help@).
If they do, you'll soon notice a heap of 'permission denied' errors being spat back either as a bounce or in mail.log, syslog, and the vacation-debug log. This is because the vacation user doesn't have access to select from the alias table in your postfix database.
To fix, you need to grant this privilege to the vacation user with an SQL query:
GRANT SELECT ON TABLE alias TO vacation;
All fixed, now email sent to an alias will send back a vacation auto-reply as well.
Making vacation autoresponder in Debian
Make the .forward file in user's home directory:
\<a href="mailto:user@email.com">user@email.com</a>
"|/usr/bin/vacation user"
Make the .vacation.msg in user's home directory:
From: <a href="mailto:user@email.com">user@email.com</a> (Joe Bloggs)
Subject: I am on Annual Leave
Delivered-By-The-Graces-Of: The Vacation program
Precedence: bulk
E-mail message goes here
cd out of the user's home directory to /home and run:
vacation -i user && chmod 644 user/.vacation.db && chown user:user user/.vacation.db
#!/bin/bash
# IP interfaces
eth0=`ifconfig eth0 | grep "inet addr" | awk '{print $2}' | cut -d: -f2`
# Change to 1 to enable logging of dropped packets
LOG=0
flush() {
iptables --flush
iptables --delete-chain
}
start() {
# Flush just in case 'start' has been called twice without stop or restart
flush
# Default policies
iptables --policy INPUT DROP
iptables --policy OUTPUT DROP
iptables --policy FORWARD DROP
# Accept all on loopback
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Accept all packets that are part of an established connection
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
# Pings
iptables -A INPUT -p icmp --icmp-type 8 -s 0/0 -d $eth0 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -p icmp --icmp-type 0 -s $eth0 -d 0/0 -m state --state ESTABLISHED,RELATED -j ACCEPT
# SSH
iptables -I INPUT -d $eth0 -p tcp --dport 22 -j ACCEPT
# HTTP
iptables -A INPUT -d $eth0 -p tcp --dport 80 -j ACCEPT
# DNS
iptables -A INPUT -d $eth0 -p udp --dport 53 -j ACCEPT
if [ $LOG -eq 1 ]; then
# Create a LOGDROP chain to log and drop packets
iptables -N LOGDROP
iptables -A LOGDROP -j LOG
iptables -A LOGDROP -j DROP
# Drop and log all other traffic inbound
iptables -A INPUT -j LOGDROP
else
# Drop all other traffic inbound
iptables -A INPUT -j DROP
fi
}
stop() {
flush
iptables --policy INPUT ACCEPT
iptables --policy OUTPUT ACCEPT
iptables --policy FORWARD ACCEPT
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
*)
;;
esac
nmap --script=smb-check-vulns --script-args=safe=1 -p445 -d (IP)
SSL:
/etc/apache2/mods-enabled/ssl.conf
SSLProtocol all -SSLv2
SSLCipherSuite HIGH:MEDIUM:!ADH
Squid
When SSL terminates at a Squid reverse proxy, instead make changes to the squid.conf likeso:
https_port 443 vhost cert=whatever.ssl key=whatever.key cafile=whatever.cer defaultsite=<a href="http://www.examplecom">www.examplecom</a> cipher=DEFAULT:!EXPORT:!LOW options=NO_SSLv2
Other Squid changes:
# PCI Verizon scan results reply_header_access X-Cache-Lookup deny all reply_header_access X-Cache deny all reply_header_access All allow all via off httpd_suppress_version_string on
PHP:
/etc/php5/apache2/php.ini
expose_php = Off
This gets rid of those horrible mountains of 'safe_mode' vulnerabilities reported to exist in versions of PHP 5.2.8 or lower, because it hides the version number. Hiding information like this that is sent back in HTTP headers is a good idea and also something else that the scan complains about.
Kind of a dodgy fix, obviously in a better world we'd be upgrading to newer version of PHP but maybe that isn't an option for whatever reason.
Don't leave any pages that call phpinfo() without checking the requestor's IP too:
if( $_SERVER['REMOTE_ADDR'] == '1.2.3.4' ) { phpinfo(); }
Apache:
/etc/apache2/apache2.conf
ServerTokens Prod
(hides apache/php/ssl versions in the footer of pages, i.e when you hit a 404, probably headers too)
In any vhost, including the 000-default, prevent TRACE with mod_rewrite
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^TRACE
RewriteRule .* - [F]
The TRACE method is an HTTP command used for debugging purposes. A client sending the TRACE command to a web server will receive an echo of the entire request, including HTTP headers. It is possible for a malicious user to obtain sensitive information from the headers, such as cookies or authentication data.
This script does an svn hotcopy of a subversion repository and then rsyncs it to an external location. Easy to script, nice trouble-free way of maintaining subversion backups.
#!/bin/bash
export RSYNC_RSH=/usr/bin/ssh
# some variables in case they change later
server=destination.com # where we are sending the backup to
user=miguel
tempcopy=/home/${user}/svnbackup
dest=/media/disk/svnserver/svnbackup
# example svn repository
# make the backup with svnadmin hotcopy
svnadmin hotcopy /data1/svn/repo/ ${tempcopy}/repo/
# rsync it to the external hard drive on my desk
rsync -aHPvz ${tempcopy}/repo/ "${user}@${server}:${dest}/repo/"
# finally, remove the backup on svnserver to free up some space
rm -rf ${tempcopy}/repo
It had never occured to me before, but suddenly I had the need to run a script on the last day of every month, in order to get the right results per month.
Given that a month's last date can be 28, 29 30 or 31, it wasn't as simple as usual!
After a bit of googling I found the easy solution. Simply get the crontab to run every day, but add into the crontab a check to see if tomorrow's date of the month is '01'. If it isn't, do nothing. If it is, then continue with the rest of the script.
36 14 * * * [ `date -d tomorrow +%d` -eq '01'] && /execute/the/real/script.sh
Edit: Hmm, got a syntax error there:
/bin/sh: -c: line 0: unexpected EOF while looking for matching ``' /bin/sh: -c: line 1: syntax error: unexpected end of file
Ah well, since I'm only executing a shell script, I just added this to my script:
#!/bin/bash
TODAY=`/bin/date +%d`
TOMORROW=`/bin/date +%d -d "1 day"`
# See if tomorrow's day is less than today's
if [ $TOMORROW -lt $TODAY ]; then
(real script executes here)
exit 0
fi
exit 1
Both articles from Sébastien Wains's blog
Named pipe
You may know “screen”, tool that can help you put your session on hold and and get back to it whenever you want.
Described below is a way to share your session with someone… this can come in handy while doing support or if you just want to share your session for some reasonYou should follow the following steps carefully
User 1 will work in the terminal
User 2 will watch what user 1 is doing1. User 1 :
mkfifo /tmp/file2. User 2 :
cat /tmp/file3. User 1 :
script -f /tmp/fileUser 1 can start working, user 2 will be able to follow his work, user 1 should type ctrl + D if wanting to stop sharing the session…
If user 2 tries to input something, the connection to the pipe will be lost and you should get back to step 2.
Screen method
This is a great way if you can’t install anything on the machine. If you are able to get “screen” installed, screen provides a much easier way, which allows all connected users to interact on the shared session. The named pipe method only allowed one user to watch what the other user was doing.
So here it goes…
1. user 1 connects to the machine and type the command “screen”
2. inside the screen terminal, user 1 hits ctrl + a and then type “:multiuser on”
3. user 2 joins in by typing “screen -x”
4. other users can join as described at point 3Both users are now sharing the session.
Ctrl + a then d will close the screen session
Links :
Source : http://linuxhelp.blogspot.com/2005/01/screen-window-manager-for-console....
VERY simple rsync script. It rsyncs a remote directory across to a local one. There's no special reason why I did this, other than I wanted to run the script on my local machine. It could just as easily rsync a local directory to a remote one.
#!/bin/sh
export RSYNC_RSH=/usr/bin/ssh
origin=remote.machine.com # the remote machine to rsync from
user=miguel # the remote user
rsync -aHPvz "${user}@${origin}:/remote/files/" "/local/files/"
I run ssh-keygen -t rsa and create a password-less key on the remote machine, and copy the .pub file into my authorized_keys. This way I can run this script on a cron, say in the middle of the night, and not have to be around to enter a password at the prompt.
Ctrl-a -> go to the start of command line
Ctrl-e -> go to the end of command line
Ctrl-p -> previous command in history
Ctrl-n -> next command in history
Ctrl-f -> next character in command line
Ctrl-b -> previous character in command line
Ctrl-r -> reverse search in history file
Ctrl-d -> delete current character
Ctrl-k -> delete from the prompt to the end of command line
Ctrl-_ -> undo (yes, but limited)
Meta-< -> go to beginning of history file
Meta-> -> go to end of history file
Meta-f -> go to next word in command line
Meta-b -> go to previous word in command line
Meta-d -> delete next word in command line (from the actual position of the prompt)
#!/bin/bash
# Basic backup script
# Written in February 2010
# Set to q to disable verbosity
VERBOSITY="v"
# The program we're using to backup
PROGRAM="rsync"
# The arguments to the program
OPTIONS="-aHP$VERBOSITY"
# The remote machine we're backing up
SERVER="max"
# The local disk to back up to
DISK="/mnt/disk"
# An array of shares on $SERVER to back up
SHARES=(
hr
finance
clients
operations
newsvn
svn
backup/jira
backup/svn/noah
backup/dbbackups/kontrol_live
backup/dbbackups/god
)
# The magic
for share in ${SHARES[*]}; do
echo "Backup of $share beginning at" `date`
$PROGRAM $OPTIONS $PROGRAM://$SERVER/$share/ $DISK/$share/
echo "Backup of $share completed at" `date`
done
Silly little script to monitor changes in a directory (new or removed files)
Takes the destination dir to monitor from stdin
One day I'll learn how to use inotify.
#!/bin/bash
# Monitor a directory for file changes
# (add or remove) as requested
touch /tmp/testdirb.$$
while true
do
find $1 -print > /tmp/testdira.$$
diff /tmp/testdira.$$ /tmp/testdirb.$$ > /tmp/dirdiff
if [ -s /tmp/dirdiff ]
then
sed -i s/"
sed -i s/">"/"Removed files"/ /tmp/dirdiff
/usr/bin/mail -s "Files changed in $1" "you@example.com"
There have been file changes in $1 since the last hour.
`cat /tmp/dirdiff`
EOF
fi
cp /tmp/testdira.$$ /tmp/testdirb.$$
sleep 3600
done
acl confickerworkinggroup dstdomain .confickerworkinggroup.org
cache deny confickerworkinggroup
mkpasswd -cl > /etc/passwd
mkgroup --local > /etc/group
http://wiki.markoschulz.de/index.php/Install_and_setup_HP_Color_LaserJet
apt-get install cupsys cupsys-bsd a2ps psutils gs-esp build-essential
wget -O foo2zjs.tar.gz http://foo2zjs.rkkda.com/foo2zjs.tar.gz
cp foo2zjs.tar.gz /usr/local/src/
cd /usr/local/src/
tar zxf foo2zjs.tar.gz
cd foo2zjs
make
make install
make cups
Add the printer
socket address is the IP of the printer
choose the 2600n driver
So easy!
Couldn't really tell you how useful this is, but it's interesting anyway.
Specifically, the article over at everythingsysadmin.com shows how you can use tcpdump to analyse what's happening over a network and potentially suss out any infected machines making too many arp who-has requests..
tcpdump -l -n arp | egrep 'arp who-has' | head -100 | awk '{ print $NF }' |sort | uniq -c | sort -n
This command displays current IP connections to the tcp/udp service.
netstat -an | grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
Rather brief notes on getting LDAP auth running on a Debian server
Install dependencies
apt-get install libpam-ldap libnss-ldap
Debconf
LDAP server Uniform Resource Identifier: ldaps://10.179.43.21/
Distinguished name of the search base: dc=badwolf,dc=greenbeedigital,dc=com,dc=au
LDAP version to use: 3
Does the LDAP database require login? No
Special LDAP privileges for root? No
Make the configuration file readable/writeable by its owner only? No
Make local root Database admin. No
Does the LDAP database require login? No
Local crypt to use when changing passwords. crypt
/etc/libnss-ldap.conf modifications
# OpenLDAP SSL mechanism
# start_tls mechanism uses the normal LDAP port, LDAPS typically 636
ssl start_tls
ssl on
/etc/ldap/ldap.conf modifications
BASE dc=badwolf,dc=greenbeedigital,dc=com,dc=au
URI ldaps://10.179.43.21/
TLS_REQCERT never
/etc/nsswitch.conf modifications
passwd: files ldap
group: files ldap
/etc/pam.d/common-auth modifications
auth [success=1 default=ignore] pam_unix.so nullok_secure
auth required pam_ldap.so use_first_pass
auth required pam_permit.so
/etc/pam.d/common-account modifications
account sufficient pam_unix.so
account required pam_ldap.so
/etc/pam.d/common-session modifications
session required pam_unix.so
session required pam_mkhomedir.so skel=/etc/skel/ umask=0022
$string = "Test test total count 500000 bytes";
$string = preg_replace('#[^0-9]#','',strip_tags($string));
echo $string;
/var/log/nagios2/nagios.log prints computer timestamps that aren't that easy to read.
Run this instead:
This script fetches a HTML page from the internet and parses the tables it finds in it, into an Excel spreadsheet.
This has been used for grabbing an AWstats index.html summary of sites and their bandwidth, and then parsing to Excel so one can tally up totals.
It's not perfect - I haven't used it in a while but from memory, it overwrites some rows due to being poorly coded :)
When the time comes that I need to use this again, I'll have another look at it.
#!/usr/bin/perl -w
use Spreadsheet::WriteExcel;
use HTML::TableExtract;
use LWP::Simple;
# Create a Table extraction
my $te = new HTML::TableExtract(gridmap=>1);
# Get the table out of my stats page
my $content = get("http://localhost/stats/index.html");
# Pass the data out of the table
$te->parse($content);
# Create a new Excel workbook
my $workbook = Spreadsheet::WriteExcel->new("totals.xls");
# Add a worksheet
my $worksheet = $workbook->add_worksheet();
# For each table, dump its parsed table rows into the worksheet
for my $ts ($te->table(1,0))
{
foreach my $row ($ts->rows)
{
$worksheet->write(0,0, $row, @{$row});
}
}
for my $ts ($te->table(1,1))
{
foreach my $row ($ts->rows)
{
$worksheet->write(1,0, $row, @{$row});
}
}
# Now add up the Total bandwidth
$worksheet->write_formula(2, 3, '=SUM(D1:D2)/1000000' );
#worksheet->write_formula(2, 3, '=SUM(D3/1000000)');
Wrote this script today using the WWW::Curl::Easy example.
Background: Recently eaccelerator screwed up a dev server so that while Apache didn't die, php scripts weren't parsing, and I got no alerts.
This script parses a php script using curl, which contains a basic 'hello()' function that prints hello out.
The perl script checks the output for 'hello' and if it doesn't find it, it sends me an e-mail (it also sends me an SMS using our Clickatell account, but I'm not giving you the credentials, silly.)
#!/usr/bin/perl
#
# Checks a php script using curl and if there is a
# problem, e-mail me
#
use strict;
use WWW::Curl::Easy;
my $url = "http://localhost/test.php";
# Init the curl session
my $curl= WWW::Curl::Easy->new() or die "curl init failed!\n";
# Give curl the URL to use
$curl->setopt(CURLOPT_URL, $url);
# a subroutine which is called for each 'chunk' as the
# file is received.
sub body_callback {
my ($chunk,$context)=@_;
# add the chunk we received to the end of the array we've been given
push @{$context}, $chunk;
return length($chunk); # OK
}
# configure which subroutine to call when some data comes in
$curl->setopt(CURLOPT_WRITEFUNCTION, \&body_callback);
my @body;
# tell the subroutine which array to put the data into
$curl->setopt(CURLOPT_FILE, \@body);
if ($curl->perform() != 0) {
print "Failed ::".$curl->errbuf."\n";
};
my $output = join("",@body);
# Check if we got 'hello' out of the php script. If we did, ignore it
if ($output=~ m/hello/i) {
}
else {
#sms me, incidentally this is done with curl too, so I create another instance of WWW::Curl::Easy
my $alert_url = "http://the-clickatell-api-command-to-sms-me";
my $curl_alert= WWW::Curl::Easy->new() or die "curl init failed!\n";
$curl_alert->setopt(CURLOPT_URL, $alert_url);
$curl_alert->setopt(CURLOPT_WRITEFUNCTION, \&body_callback);
$curl_alert->setopt(CURLOPT_FILE, \@body);
if ($curl_alert->perform() != 0) {
print "Failed ::".$curl_alert->errbuf."\n";
};
# send me an e-mail now, using Sendmail
unless(open (MAIL, "|/usr/sbin/sendmail -t")) {
print "error.\n";
warn "Error starting sendmail: $!";
}
else {
print MAIL "From: web\@the-server.com\n";
print MAIL "To: miguel\@example.com\n";
print MAIL "Subject: PHP problem on the server\n\n";
print MAIL "The server has stopped parsing php! Panic! Panic!";
close(MAIL) || warn "Error closing mail: $!";
}
}
#!/bin/perl
# Build an array of e-mail addresses to be Bcc'd to
my @bcc_addresses = (
'foo@bar.com',
'foo@bar2.com');
# The main recipient
my $to='foofoo@bar.com';
# Join the bcc addresses from the array, separate with comma
my $bcc = join(", ", @bcc_addresses);
# Where it's coming from
my $from='miguel@foobar.com';
# e-mail Subject
my $subject="Perl mail example";
# The e-mail content follows
my $out = "Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Nullam vestibulum dictum lectus.
Etiam at sapien. Donec fermentum dictum nisi. In ornare adipiscing massa.";
# Now let's fire up sendmail and push the data into it, then send
open(MAIL, "|/usr/sbin/sendmail -t");
print MAIL "To: $to\n";
print MAIL "Bcc: $bcc\n";
print MAIL "From: $from\n";
print MAIL "Subject: $subject\n";
print MAIL $out;
close(MAIL);
On my Etch servers, proftpd's logs in /var/log/proftpd/ are all mixed up. Just logging in, transferring a file, logging out, each action or series of actions were often being logged with mixed dates and times, never in order, and always wrong.
Turns out by default proftp is logging times in GMT or somesuch, I guess based on the client's system? To force proftpd to log by the server's local time, add these two lines to /etc/proftpd/proftpd.conf
TimesGMT off SetEnv TZ :/etc/localtime
Should be all good now.
These are possibly my worst written notes ever below. Beware!
Xen was remarkably easy to install on Lenny. I just installed the latest xen kernel
(2.6.26-2-xen-686) and xen-hypervisor-3.2-1-i386 xen-utils (3.0?), also these bridge-utils
iproute sysfsutils libc6-xen xen-tools
Made sure these were in /etc/xen/xend-config.sxp
(network-script network-bridge)<br>
(vif-script vif-bridge)<br>
/etc/xen-tools/xen-tools.conf was basically this:
dir = /opt/xen #mkdir'd this first
install-method = debootstrap
size = 4Gb # Disk image size.
memory = 128Mb # Memory size
swap = 128Mb # Swap size
# noswap = 1 # Don't use swap at all for the new system.
fs = ext3 # use the EXT3 filesystem for the disk image.
dist = lenny # Default distribution to install.
image = sparse # Specify sparse vs. full disk images.
gateway = 192.168.1.254
netmask = 255.255.255.0
broadcast = 192.168.1.255
passwd = 1
mirror = <a href="http://ftp.us.debian.org/debian/">http://ftp.us.debian.org/debian/
[/geshifilter-code]
After creating the DomU and firing it up, couldn't get a login prompt at the console, and ssh'ing in gave this error when logging in:
stdin is not a tty
"Additional note for domU on lenny using xen-tools
xen-tools don't use hvc0 as the console device in /etc/inittab and don't install udev (leading to /dev/pts missing in domU).
This makes logging in via xm console and via ssh impossible, because getty doesn't have a proper console to attach to and ssh can't attach to a pseudo terminal.
To fix this,
1. add to /etc/xen-tools/xen-tools.cfg:serial_device = hvc0"
So added that line to /etc/xen-tools/xen-tools.cfg as required.
Also made these changes to /boot/grub/menu.lst on the Dom0 first:
title Xen 3.2-1-i386 / Debian
GNU/Linux, kernel 2.6.26-2-xen-686
root (hd0,0)
kernel /boot/xen-3.2-1-i386.gz
com1=9600,8n1 console=com1,vga
module /boot/vmlinuz-2.6.26-2-xen-686
root=/dev/sda1 ro console=tty0 console=hvc0
module /boot/initrd.img-2.6.26-2-xen-686
Changes being the console and com1 lines.
From the same notes above: reason is 'To get output from grub, XEN, the kernel and getty (login prompt) via both vga and serial console to work'.
Also modified /etc/inittab so it included this
1:2345:respawn:/sbin/getty 38400 hvc0<br>
2:23:respawn:/sbin/getty 38400 tty1<br>
From the same notes above, 'The tty1 will show up at the vga output, and the hvc0 will show up at the serial console.'
Then after a reboot of the Dom0, I could create new DomU like this:
xen-create-image --hostname=HOSTNAME --ip=IP --passwd --role udev<br>
and it would actually install udev properly, no having to fiddle with /etc/fstab with devpts like mentioned here, or having to install udev while mounting the image locally and chrooted etc etc etc. With the grub, inittab and xen-tools changes above, along with the '--role udev' when creating new images, it 'just works' (tm) both via console and ssh.
Other thing I did was quickly install openntpd on the DomU as this removed the 'clock went backwards' errors flooding the logs/console.
(edit : apparently this is how to deal with those 'clock backwards' errors)
Short story: I'm liking Xen a lot! And not just because Dom0 reminds me of this guy
