Powershell Invoke-WebRequest to Jenkins

Scenario: Jenkins with authentication turned on and a user wanting to use Powershell to remotely execute a job

These 2 links provide most of the background and give examples for WGET. Jenkins authentication being managed by Crowd in this scenario

https://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API
https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients


Invoke-WebRequest -Uri $url -Headers @{"Authorization" = "Basic "+[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($User+":"+$Password ))}

Linux: Checking Disk Space

Scenario: The monitoring system sends an email stating a Linux server is running low on disk space. I often had a hard time figuring out what part of the filesystem was growing and needed attention. I've used this a few times with good success.

cd /path/to/some/where
du -hsx * | sort -rh | head -10

Keep drilling down to find the root cause of the disk space issue.

Jenkins Slave Automatic Start

Scenario: Jenkins Master running on Windows with a Jenkins Slave running on CentOS

Problem: The Jenkins Slave is not automatically connecting after a restart of the slave server, master server, or master Jenkins service

Solution: In early 2013 Jenkins introduced a security token fix outlined here

The thread provides many options for connecting. I decided the path of least resistence was to enable anonymous connect/disconnect for slaves. This is acceptable in my environment because each Jenkins instance is behind a firewall and I feel the security risk is minimal. This might not always be the case, but in this situation, it fixed the problem quickly.

I created a jenkins-slave service in /etc/init.d based on this article. I made a few modifications to the script, but it's working great.

AWS SimpleDB IAM Access

We have nodejs applications running in AWS EC2. Instances should be disposable. We setup the apps to log to AWS SimpleDB. But creating a user to access SimpleDB was more difficult then I expected. In IAM, I created a user and applied two policies using the policy generator:

{
  "Statement": [
    {
      "Action": [
        "sdb:*"
      ],
      "Resource": [
        "arn:aws:sdb:*:1111222333:domain/node*"
      ],
      "Effect": "Allow"
    }
  ]
}

{
  "Statement": [
    {
      "Action": [
        "sdb:ListDomains"
      ],
      "Resource": [
        "arn:aws:sdb:*:1111222333:*"
      ],
      "Effect": "Allow"
    }
  ]
}

The number is the AWS account number
The key for success was the second policy for Listing Domains

This is an excellent way to connect and manage SimpleDB from the Chrome browser
https://chrome.google.com/webstore/detail/sdbnavigator/ddhigekdfabonefhiildaiccafacphgg?hl=en-US

Statsd + Graphite Server Configuration

Thanks to Eric for most of this work, he did the heavy lifting and then I documented https://github.com/etsy/statsd/

Yum
yum install python-pip graphite-web python-carbon python-whisper graphite nodejs npm mysql-server mysql-client MySQL-python

User
Create user nodejs to run application
vi /etc/sudoers
Add: nodejs ALL = NOPASSWD: /usr/bin/node
Comment: #Defaults requiretty

Node
mkdir /opt/statsd
cd /opt/statsd
npm install nodeunit
npm install temp
npm install underscore
chown -R nodejs:nodejs /opt/statsd

Mysql
mysql -u root -p

create database graphite;
create user 'graphite'@'localhost' identified by 'password';
grant all on graphite.* to 'graphite'@'localhost';

cd /etc/graphite-web
Edit local_settings.py
Change Database Configuration settings, uncomment and edit to the following

DATABASES = {
    'default': {
        'NAME': 'graphite',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'graphite',
        'PASSWORD': 'password',
        'HOST': 'localhost',
        'PORT': '3306'
    }
}

Graphite
Run the following
/usr/lib/python2.6/site-packages/graphite/manage.py syncdb
/usr/lib/python2.6/site-packages/graphite/manage.py createsuperuser

Graphite will connect to Apache by default to root directory
service httpd restart
Open Browser and go to http:// and should see graphite

Statsd Config Download statsd from Github and extract to /opt/statsd

Copy exampleConfig.js to Config.js and add the following

{
  "graphitePort": 2003,
  "graphiteHost": "127.0.0.1",
  "address": "0.0.0.0",
  "port": 8125,
  "flushInterval": 10000,
  "deleteIdleStats": false,
  "deleteGauges": false,
  "deleteTimers": false,
  "deleteSets": false,
  "deleteCounters": false,
  "graphite": {
    "legacyNamespace": true,
    "globalPrefix": "stats",
    "prefixCounter": "counters",
    "prefixTimer": "timers",
    "prefixGauge": "gauges",
    "prefixSet": "sets"
  }

Firewall
Open UPD port in ipTables/Firewall
iptables -I INPUT 10 -m state --state NEW -p udp --dport 8125 -j ACCEPT (10 is the line number, change per ipTables open ports)
/sbin/service iptables save
service iptables restart

Upstart
Create /etc/init/statsd

#!upstart
description "statsd"

env PROGRAM_NAME="statsd"
env FULL_PATH="/opt/statsd"
env FILE_NAME="stats.js"
env NODE_PATH="/usr/bin/node"
env USERNAME="nodejs"

start on runlevel [2345]
stop on runlevel [016]

script
    echo $$ > /var/run/$PROGRAM_NAME.pid
    cd $FULL_PATH       
    exec sudo -u $USERNAME $NODE_PATH $FULL_PATH/$FILE_NAME /opt/statsd/dbs2Config.js >> /var/log/$PROGRAM_NAME.sys.log 2>&1
end script

Start statsd
initctl start statsd

Cacti LDAP Authentication with Active Directory

Cacti 0.8.8a
Windows 2008 R2 Domain Level

yum install php-ldap

Settings -> Authentication -> LDAP Authentication

LDAP used for admins only, graph readers will use the guest account

Guest User: No User
User Template: admin
Server: FQDN of domain controller
Port Standard: 389
Port SSL: 636
Protocol Version: Version 3
Referrals: Disabled
Mode: Specific Searching
Distinguished Name: Blank
Require Group Membership: Not Checked
Group Distinguished Name: CN=cacti_admins,OU=groups_users,DC=foo,DC=domain,DC=com
Group Member Attribute: member
Group Member Type: Distinguished Name
Search Base: ou=groups_users,DC=foo,DC=domain,DC=com
Search Filter: (&(objectclass=user)(objectcategory=user)(userPrincipalName=*))
Search Distinguished Name: ldaplookupuser@foo.domain.com
Search Password: *********

Tips: Some users report issues trying to connect via local and LDAP in the same browser session, log out to test

Execute MS SQL scripts from Jenkins with Powershell

Prerequisite: Install SQL Native Client and SQL Command Line Tools on Jenkins server

This script will look inside a directory and sort the .sql scripts by name. It will execute each script and right the output to a temp file. Then it displays the output on the Jenkins console.

Options for the sqlcmd.exe are here: MSDN sqlcmd Utility

ECHO "Executing SQL Scripts"
$sqldir = "$tagDir\"
$sqlfiles = Get-ChildItem $sqldir *.sql -rec | Sort-Object Name 
foreach ($fileName in $sqlfiles) {
$file = $sqldir + $fileName
& "C:\Program Files\Microsoft SQL Server\100\Tools\Binn\sqlcmd.exe" -S $sqlserver -U $deployuser -P $deploypassword -d $database -i $file -e -b -o C:\temp\output.txt
Get-Content "C:\temp\output.txt"
}

This code will make a backup of a database

& "C:\Program Files\Microsoft SQL Server\100\Tools\Binn\sqlcmd.exe" -S $sqlserver -U $deployuser -P $deploypassword -Q "BACKUP DATABASE $database TO DISK='$backupPostLocation'"