20150214

Improving My Web Browsing (Linux)

DISCLAIMER: This project is posted for information only and should take no more than 2 hours to complete. Only an experienced Linux user should attempt this task due to the work involved. It requires being comfortable at the command line and downloading packages from the Internet.

Overall, I rate this project as "advanced" since an experienced Linux user will be able to understand what I planned to do and implement it. YMMV

In this project, I worked at getting polipo to work with pdnsd. My intention was to speed up polipo, the caching web proxy server. Information on polipo is found by entering apt-cache to show its description:

$ apt-cache show polipo

Originally I started with polipo, and noticed that Iceweasel was lofting and wait times while loading web pages was over 15 seconds. After searching on the Internet for "speed up polipo" I happened on a post on the crunchbang.org forum by "hardran3" entitled '"Faster" browsing with polipo & pdnsd.'

Polipo is configured to show:
  • its menu at http://127.0.0.1:8123/polipo/
  • its status at http://127.0.0.1:8123/polipo/status?
  • its current configuration at http://127.0.0.1:8123/polipo/config?
  • its known servers at http://127.0.0.1:8123/polipo/servers? and
  • and its disk cache index at http://127.0.0.1:8123/polipo/index?
As instructed on the tips page, I installed pdnsd using aptitude: $ sudo aptitude install pdnsd

Pdnsd is the proxy DNS server with permanent caching. More information on it is found by entering apt-cache to show its description: $ apt-cache show pdnsd

After installing polipo, I edited /etc/polipo/config without ad-blocking and allowed the amount of information leaked by commenting the lines for censoredHeaders and censorReferer.

In this way it was adapted for Crunchbang Linux on my desktop PC which is directly connected to the Internet via aDSL modem / router.

Next, /etc/pdnsd.conf was edited using the configuration shown. However, the original text for pdnds.conf was missing a "}", so the last six lines of the code given should be:

48:source {
49: ttl=86400;
50: owner="localhost.";
51: serve_aliases=on;
52: file="/etc/hosts";
53: }


I also opted to keep the following from /etc/pdnsd.conf:

55:rr {
56: name=localhost;
57: reverse=on;
58: a=127.0.0.1;
59: owner=localhost;
60: soa=localhost,root.localhost,42,86400,900,86400,86400;
61:}


However, the above 7 lines are optional. Finally I edited the /etc/dhcp/dhclient.conf file as directed.

In order to get pdnsd to run, the file /etc/default/pdnsd also needs to be edited as follows:

01: # do we start pdnsd ?
02: START_DAEMON=yes


* Note that I prefaced lines in the respective configuration files with line numbers. This is for information purposes and should be left out in each file.

To get dhclient to restart, unplug the network cable and plug it back in. If it still isn't reset, do not worry. Just start up polipo with: $ sudo /etc/init.d/polipo restart

The system will respond with: Restarting polipo: polipo. ...and start up pdnsd with:
$ sudo /etc/init.d/pdnsd restart

To which the system will respond with: [ ok ] Restarting pdnsd (via systemctl): pdnsd.service.

To check the status of and ensure pdnsd is running: $ sudo /etc/init.d/pdnsd status
The system will respond with:  

● pdnsd.service - LSB: Start pdnsd
   Loaded: loaded (/etc/init.d/pdnsd)
  Drop-In: /run/systemd/generator/pdnsd.service.d
           └─50-pdnsd-$named.conf
   Active: active (running) since Sat 2015-02-14 01:17:28 PST; 2h 42min ago
  Process: 24149 ExecStop=/etc/init.d/pdnsd stop (code=exited, status=0/SUCCESS)
  Process: 24156 ExecStart=/etc/init.d/pdnsd start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/pdnsd.service
           └─24161 /usr/sbin/pdnsd --daemon -p /var/run/pdnsd.pid


Once pdnsd is running, polipo is using 127.0.0.1 as the address for the internal DNS server. Overall, the time spent debugging was due to pdnsd not starting up.

Once it started running, I got the following response using netstat: $ sudo netstat -atunp | grep "127.0.0.1:53"




Results:

Internet response times are improved. Most of the webpages are cached permanently in /var/cache/polipo and the DNS information in /var/cache/pdnsd - this allows for faster access to webpages because they are permanently stored in cache.

Update; 20150411

I have also added the polipo and pdnsd packages to my netbook which runs Mint 17.1.


Sometimes I get one webpage that never resolves on my netbook. After careful research of the issue (504 Gateway error) led me to reinstall resolvconf. Then I added GoogleDNS to server section in /etc/pdsnd.conf to improve lag time:

server {
label = "GoogleDNS";
ip=8.8.8.8;
proxy_only=on;
timeout=10;
}

After forcing a reload of pdnsd, I could access the webpage.


Kudos: Thanks to Hardran3 for the tips on getting polipo to work with pdnsd, and especially to Jiri H for developing Crunchbang Linux.

While development of Crunchbang Linux is finished, I will still use it because it still works.

Reference:

Caching Web Proxy and DNS Server pictures:

Crunchbang:
Index
» Tips, Tricks & Scripts
» "Faster" browsing with polipo & pdnsd

http://crunchbang.org/forums/viewtopic.php?id=11881

Mint Linux is derived from Ubuntu, and the following link helped with resolve the 504 Gateway error:

http://www.n00bsonubuntu.net/content/speed-up-your-web-browsing-by-caching-dns-to-your-hard-drive-in-ubuntu/

No comments: