Raspberry Pi Personal Hotspot - Squid with Adzapper Config (part 2)

-- Link to Part 1 --

Here are the steps I used to finish my personal wifi hotspot. I was able to find lots of tutorials that show how to make a simple wifi to ethernet bridge, but I wanted to save the max bandwidth possible. I'll be using adzapper and a squid3 cache for bandwidth saving. I don't usually block ads, but since this is a metered internet connection I don't like the thought of paying for ads. Squid can be configured for very aggressive caching, but I have left it on the default configuration. 

Inside view of the "PiSpot". The video and audio port have been removed to save space.

Battery, 4 port USB hub, 4G dongle, and a shortened USB Cable.

Here we see the fully operational battle station -- err, PiSpot.

You can see the various components in the pictures above. I removed the plastic casings to save space. I haven't done any testing on the battery life, but it should last at least a few hours with light traffic. Here are the parts I used:

• Raspberry Pi Model B 1st generation (256MB RAM) --Model A would work as well
• 4 port USB 2.0 Hub - Iogear Model GUH285 -- I chose this because of its size and it was <$10 at Fry's.
• EasyACC BP8400 Power Bank 5600mAh Battery - Amazon Link
• Belkin F9L1005 Wifi Adapter (rtl8192cu)
• FreedomPop 4G Adapter - Amazon Link

TODO:
Charge battery without opening case.
Power button so the the unit can be turned on or off without opening the case.

This post will assume that you are already able to connect to the Raspberry Pi WiFi network that was created in part 1.

Install Software

Make sure squid and adzapper are installed

apt-get install squid3 adzapper

Configure Squid

First make sure that the pi is configured for ip forwarding at /etc/systcl.conf. Uncomment or add:

net.ipv4.ip_forward=1

Edit /etc/squid3/squid.conf and to something similar to the config below. This config includes lines to enable adzapper and transparent proxy. Max storage size is 1.5GB. Make sure to change the IP address to your network.  **I'm sure this can be fine tuned for better bandwidth savings, let me know if you have any suggestions!

cache_mgr dustin
cachemgr_passwd dustin all
redirect_program /usr/bin/adzapper.wrapper
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 192.168.254.0/24 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
http_access deny all
http_port 3128 transparent
 cache_mem 128 MB
cache_dir ufs /var/spool/squid3 1500 16 256
coredump_dir /var/spool/squid3
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

Configure Adzapper

/etc/adzapper.conf should look something like this:

ZAP_MODE=""

ZAP_BASE=http://adzapper.sourceforge.net/zaps

ZAP_BASE_SSL=https://adzapper.sourceforge.net/zaps # this can probably be ignored

ZAP_PREMATCH=

ZAP_POSTMATCH=

STUBURL_AD=$ZAP_BASE/ad.gif

STUBURL_ADSSL=$ZAP_BASE_SSL/ad.gif

STUBURL_ADBG=$ZAP_BASE/adbg.gif

STUBURL_ADJS=$ZAP_BASE/no-op.js

STUBURL_ADHTML=$ZAP_BASE/no-op.html

STUBURL_ADMP3=$ZAP_BASE/ad.mp3

STUBURL_ADPOPUP=$ZAP_BASE/closepopup.html

STUBURL_ADSWF=$ZAP_BASE/ad.swf

STUBURL_COUNTER=$ZAP_BASE/counter.gif

STUBURL_COUNTERJS=$ZAP_BASE/no-op-counter.js

STUBURL_WEBBUG=$ZAP_BASE/webbug.gif

STUBURL_WEBBUGJS=$ZAP_BASE/webbug.js

Now iptables needs to be configured to route traffic through squid. Create a new file:

nano iptables.sh

Add the the rules below. Careful with line breaks when cut/pasting, there should only be 5 lines total.

#nat for wifi
iptables -t nat -A POSTROUTING -j MASQUERADE
#squid transparent cache
iptables -t nat -A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.254.1:3128
iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128

Hit ctrl+o to save the file, then ctrl-x to exit. Now the file needs to be made executable and copied to an appropriate location.

chmod +x iptables.sh
sudo cp iptables.sh /etc/init.d/

Apply the configuration at boot.

sudo update-rc.d iptables.sh start 99

That should wrap it up. At this point I suggest doing a power cycle on the Pi to make sure everything comes up automatically. 

Raspberry Pi Personal WiFi Hotspot (part 1)

I recently bought a Nexus 7 16GB WiFi with the hopes that it could replace my aging and heavy laptop. My first thought for remote connectivity was to use the personal hotspot feature on my phone... until I saw the prices. $20/month on top of what I already pay for a service I will rarely use? No thanks! Here's what I did instead. I already had everything except for the 4G* usb adapter I bought off of Amazon. 

*The 4G service is Clearwire wimax (now owned by Sprint). Sprint is replacing wimax with LTE, but will be keeping the wimax online until sometime in 2015. 

Equipment
- Raspberry Pi - Model A will work for this project, but I only have a Model B. 
- 4GB or greater SDCard 
- 1A or greater micro usb phone charger. I have successfully used the stock HTC charger, and a generic 1 amp car charger with both USB devices connected directly to the  pi.
- USB WiFi Adapter - I used a Belkin F9L1005 (rtl8192cu) - I DO NOT recommend using this model!
- USB 4G Adapter - I am using a FreedomPop branded adapter which appears to be this: http://www.ubeeinteractive.com/products/mobility/ubee-4g-wimax-usb-pxu-1960. I purchased one from amazon here: http://www.amazon.com/Freedom-Stick-Bolt-Modem-Black/dp/B009FCGASA/ref=sr_1_3?s=wireless&ie=UTF8&qid=1377739418&sr=1-3&keywords=freedompop

There are undoubtedly cheaper ways to get internet connectivity, you could probably buy a MiFi device for cheaper than the total cost of this setup, but I have another project in mind that I plan on using the same Raspberry Pi for. 

Part 1 - Prepare the Raspberry Pi OS

Start with the official Rasbian distro.  Instructions and the download can be found here: http://www.raspberrypi.org/downloads

Run the raspi-config utility and configure your desired settings. 
* Set your password
* Expand Filesystem
* Disable boot to desktop
* Enable SSH
* Use the 16MB memory split if possible. I had some problems booting my revision 1 pi on the 16MB setting and had to go with 32MB.

Reboot and update assuming you are plugged into a network.
sudo apt-get update 
sudo apt-get upgrade 

Remove the Desktop environment for a little extra space on the SD card. The only reason for this is to allow a larger squid disk cache size.
apt-get remove --auto-remove --purge libx11-.*
apt-get auto-remove

Install the software we will be using.
apt-get install hostapd hostapd-utils dnsmasq squid3 adzapper

At this point the software components need to be configured.

Part 2 - Configure WiFi AP

Plug in your wifi adapter and make sure the system recognizes it by running ifconfig. It will probably be listed as wlan0.
Configure the interfaces at /etc/network/interfaces. The sample below includes dhcp setting for eth1, which will be the USB 4G adapter.

auto lo

iface lo inet loopback
iface eth0 inet dhcp

iface eth1 inet dhcp

iface wlan0 inet static
address 192.168.254.1
netmask 255.255.255.0

Configure hostapd. If using the rtl8192cu usb adapter then you will need a custom hostapd binary available here: http://blog.sip2serve.com/post/38010690418/raspberry-pi-access-point-using-rtl8192cu
Edit /etc/hostapd/hostapd.conf to something like the sample below. 

interface=wlan0
driver=rtl871xdrv
country_code=US
ssid=Mobile-Wifi
hw_mode=g
channel=1
wpa=2
wpa_passphrase=Your_Password
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
auth_algs=2
macaddr_acl=0

Configure dnsmasq as a dhcp server. Create a new /etc/dnsmasq.conf file or add the following sample to the top of the existing one. This specifies the dhcp pool as 192.168.254.100-200 with a 12 hour lease.

domain-needed
interface=wlan0
domain=mobile-wifi.local
dhcp-range=192.168.254.100,192.168.254.200,255.255.255.0,12h

Now we need to make the pi act as a gateway and forward packets. This is just a matter of uncommenting a line in /etc/sysctl.conf. Add or uncomment:
net.ipv4.ip_forward=1


That should be all that's needed for a simple AP. Shut down the pi and plug in the USB 4G adapter. Start it back up and attempt to connect to the wifi. If connected, open an ssh connection to 192.168.254.1 and see if you can ping google.
That's all I have time for at the moment. I'll work on posting a quick howto for saving a little bandwidth using squid and adzapper. 

Raspberry Pi with Wifi and 4G USB adapter.

ntop via svn on Ubuntu Server 12.04

Install ntop svn on Ubuntu Server 12.04

Quick post on installing the most up to date development version of ntop on Ubuntu server. The version of ntop in the official repositories is 4.x, which will show as being out of date in the web gui.The development version will show as unstable.

1. Install prerequisites:

sudo apt-get install libpcap-dev libgdbm-dev zlib1g-dev librrd-dev python-dev libgeoip-dev subversion

2. Download software:

cd ~
svn co https://svn.ntop.org/svn/ntop/trunk/ntop/

3. Configure source:

cd ntop
./autogen.sh

4. Compile source:

make

5. Install:

sudo make install

6. Configure ntop. Without ldconfig you will get an error "ntop: error while loading shared libraries: libntopreport-5.0.2.so: cannot open shared object file: No such file or directory"

sudo ldconfig
sudo ntop --set-admin-password

7. Configure permissions:

sudo chown -R nobody:nogroup /usr/local/var/ntop/

8. Start ntop:

sudo ntop -d

9.  Access ntop:

http://localhost:3000

Done!!

Quick and Dirty: Set up a Zimbra Email Server

This guide focuses on setting up an email server ideal for home or small business use. Setup is extremely quick and very low cost.

What I mean by "Quick and Dirty" is that in most cases the default settings will be used. Completing each step in this guide should get you a fully function server in a few hours (depending on hardware and download speeds. Sometimes defaults maybe not optimal or most secure, but the system will be running in the end.

Zimbra offers a full featured email solution that does not require a super-powered server. (if a low amount of users are accessing it) I have been successfully using the latest version on my home server (only 2 users) configured on a VM with 768MB RAM and 1 AMD Athlon64 3000+ processor. Although it takes a while for all the services to start at boot, I  haven't had any performance issues while using the system. I chose the Zimbra system because of its ease of setup and powerful features.

Here are the materials I used for this setup:

• 768MB RAM (1GB or higher recommended)
• Athlon64 3000+ (32 bit can be used, but is being phased out)
• Comcast internet connection with DHCP IP address
• Postini Anti-Spam service used as smarthost and general spam protection, This is necessary for a server with a non-static WAN IP address.
• CentOS 5.5 x64

** This guide will not be covering configuring Postini or MX records.**

Install and Configure CentOS 5.5

CentOS 5.5 can be downloaded at www.centos.org. Burn the ISO or point your virtual machine to it. Only CD 1 will be needed.

Follow the installer directions. When the Network devices screen appears, make sure to change the network device to static IP.

- Disable IPv6 if not being used on your network.

- Set "Hostname" to manually and configure the FQDN that your server will be using. (e.g. mail.somename.com) The hostname is checked when setting up Zimbra, and it's easiest to configure it at this screen, so make sure it's correct if possible.

- Configure the Gateway and DNS as apporpriate for your network.

Click next, and assign the timezone and root password as needed on the next screens.

When the software selection screen is reached, make sure that all desktop options are unchecked, and the "Customize now" button is selected.

Remove Gnome and choose customize so that only Disc 1 is needed.

At the next screen, go through all the categories on the left, and uncheck everything EXCEPT "Base" in the "Base System" category.

The only box that should be checked is "Base"

Once configured, click "Next" and proceed with installing the system.

All additional needed software will be installed via yum and wget once the installation completes.

When the install finishes and the server finishes booting the first time, the setup wizard will appear. Since this server will be behind a physical firewall on a trusted network, I will be disabling the software firewall and SELinux options. 

Disable firewall and SELinux

Use the above settings while installing. SELinux is not advised. Firewall settings can be tested and configured after the installation if desired.

Once configured, select OK.

"Sendmail" should be disabled, as Zimbra uses it's own integrated MTA.

- Select "System Services"

- Scroll down the list until "sendmail" is displayed and uncheck it with the spacebar.

Disable sendmail

After configuring, press "OK" and then "QUIT", and then reboot the computer. 

# reboot

After rebooting, make sure the date and time is correct.

# date

If not correct, use:

# date MMDDhhmmYYYY

Example: "date 102015302010" will change the date and time to Oct. 20th, 3:30pm, 2010. It is recommended to sync time with an NTP server, covered elsewhere.

When the date is correct, update the system:

# yum update -y

This will install all security updates and update the kernel, which will require a reboot when finished.
After the reboot, Zimbra its prerequisites can be downloaded and installed.

Installing Zimbra

 Zimbra requires a few packages (sysstat and gmp) that aren't included with the basic CentOS install.

# yum install sysstat gmp -y

Now the hosts file needs to be changed because we are using a static IP with a smart host.
Edit /etc/hosts and change the IP address on the last line to the LAN IP of the server. It should look something like this:

Reconfigure /etc/hosts

The IP address above was initially the public WAN IP of the router.

The Zimbra software package can be downloaded here: http://www.zimbra.com/downloads/os-downloads.html 

Locate the latest release for Red Hat Enterprise Linux 5. Right click the link for the appropriate architecture and copy the link location.

The easiest way is usually to SSH to the server and then use wget to download the package. Example:

wget http://files2.zimbra.com/downloads/6.0.10_GA/zcs-6.0.10_GA_2692.RHEL5_64.20101215170845.tgz

When the download is complete, unpack it:

# tar xvzf zcs-* (tab)

Change to the newly created directory and run the install script:

# ./install.sh --platform-override

The first portion of the install is self explanatory. Accept the license agreement and install all the default components.
Be sure to read the output, some of the defaults will exit the installer if not changed. (License agreement, Platform override, and modifying the system)

At one point during the install the following error will appear:

DNS ERROR resolving MX for mail.somename.com
It is suggested that the domain name have an MX record configured in DNS
Change domain name? [Yes]

If you are using a smarthost with a dynamic IP, than type "No".

The following configuration screen will appear when the files are finished copying:

Zimbra installer config screen

For this guide, everything will be left at default, however the admin password needs to be changed.
- Press "3" and then enter
- Press "4" and then enter
- Enter the desired admin password and press enter
- Press "r" and then enter
- Press "a" and then enter
Accept the default entries and then type "y" when asked to modify the system.
After the applying the settings and starting services, the installer will exit and the system should be ready for login.

Configuring Zimbra

Log into the admin page at https://YourIPAddress:7071

The username will be "admin" and the password will be what was specified earlier. 

First, create a new account.

- In the categories on the left panel, click "Accounts," directly under "Addresses."

- Click "New" and then enter the desired info. 

Smarthost and open relay now needs to be configured.
- Click the "Servers" category on the left panel.
- Select your server under the "host name" category in the middle and then click the "edit" button towards the top.
- Click the "MTA" tab

Zimbra web config screen

- Verify the host name is correct in the first field
- Enter your smarthost address in the second port. This will be something like outbounds#.obsmtp.com for Postini spam filtering.
- Enter the trusted smarthost network in the "MTA Trusted Networks" field. For Postini, this will be: 74.125.148.0/22
- Save changes and exit the admin console
- Uncheck the "DNS Lookups" box

That should be it. The system should now be able to send and receive mail to the smarthost/spam filter, providing the MX records and smarthost are configured correctly.