moved from https://github.com/epsylon/Border-Check

README.md

@@ -1,3 +1,158 @@
-# Border-Check
+Border Check v0.2 - 2015
+========================
+
+Border Check (BC) is a browser extension that illustrates the physical and political realities of the internet's infrastructure using free software tools.
+
+As one surfs the net, data packets are sent from the user's computer to the target server. These data packets go on a journey hopping from server to server, potentially crossing multiple countries, until the packets reach the desired website. In each of the countries that are passed different laws and practices can apply to the data, influencing whether or not authorities can inspect, store or modify that data.
+
+In realtime BC lets you know which countries you surf through as you browse the web. Additionally BC will illustrate this process on a world map and (where available) provide you with contextualizing information on that country's laws and practices regarding your data.
+
+BC runs on OSx and Unix systems.
+
+Currently supporting the following browsers on OSX and Unix systems: Firefox, Chromium, Chrome, Safari
+
+NOTE: Browser history has to be enabled.
+
+
+### Installing BC:
+
+BC v0.2 (2015) provides a "Wizard" installer that makes installing the required libraries and packages for BC easier. 
+
+The wizard runs automatically the first time you run BC. It will check if your system has all the dependencies met for BC and if not the wizard will download and install them automagically. During the wizard BC will also install the required maps and databases.
+
+Border Check has the following dependencies:
+
+### Python:
+
+* [pygeoip](https://pypi.python.org/pypi/pygeoip/0.2.7) 
+* [lxml](https://pypi.python.org/pypi/lxml/3.2.3)
+* [biplist](https://pypi.python.org/pypi/biplist/0.5) (if you plan to use Safari)
+
+For **Debian**-based systems (ex: Ubuntu), first run:
+
+`sudo apt-get install python-pip` to install [pip](https://pypi.python.org/pypi/pip/), python's super usefull package manager.
+
+If you already have pip: `pip install lxml` and `pip install pygeoip`
+
+On **OSX** it's recommended you install [Homebrew](http://brew.sh/) first and use that to get [python + pip](https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python).
+
+
+### LFT
+
+BC uses [LFT v3.73 (08/2014)](http://pwhois.org/lft/) for tracerouting.
+
+On **Debian** and **Ubuntu** it needs to be built from source and requires libpcap [`sudo apt-get install python-libpcap`]
+
+To make this process more easy, BC (v0.2 2015) provides you a binary with the source. It is called `lft.linux` and is on folder `bin`.
+
+On **OSX** you can use Homebrew: `brew install lft` which will automatically download and build the package.
+
+#### Note on provided LFT binary
+
+We do provide a static compiled binary of lft in the bin directory.
+
+Please consider that this is for convenience only, you are invited to make your own.
+
+
+### GeoIP databases and js libraries
+
+BC will automatically unpack the newest geoip databases on the first run. 
+
+Javascript mapping libraries are included in the package.
+
+
+## Options and features:
+
+See the included examples for usage.
+
+```
+bc [OPTIONS] 
+
+  --version             show program's version number and exit
+  -h, --help            show this help message and exit
+  -d, --debug           debug mode
+  -l LFT_PATH           path to lft (fetch from source or use provided binary)
+  --xml=EXPORT_XML      export traces to xml (ex: --xml foo.xml)
+  --load=IMPORT_XML     import traces (non root required) (ex: --load bar.xml)
+  --bh=BROWSER_HISTORY  set browser's history path
+  -b BROWSER            set browser type to be used: F = Firefox / C = Chrome / S = Safari / Ch = Chromium / N = None
+
+```
+
+#### Examples of usage:
+
+Normal BC run:
+
+`$ python bc`
+
+Debug mode:
+
+`$ python bc --debug`
+
+Export 'tracing' results to xml:
+
+`$ python bc --xml "mytravel.xml"`
+
+
+#### More options (set browsers, paths, etc):
+
+Use Firefox browser: 
+
+`$ python bc -b F`
+
+Use Chromium browser: 
+
+`$ python bc -b Ch` 
+
+Set browser's history path manually, on Galeon/Unix:
+
+`$ python bc --bh ".galeon/mozilla/galeon/places.sqlite"`
+
+Set browser's history path manually, on Chrome/OSx:
+
+`$ python bc --bh "Library/Application Support/Google/Chrome/Default/History"`
+
+Set browser's history path manually, on Safari/OSx:
+
+`$ python bc --bh "Library/Safari/History.plist"` 
+
+Set browser's history path manually, on Firefox/Unix:
+
+`$ python bc --bh "Library/Safari/History.plist"` 
+
+Import 'traces' from xml:
+
+`$ python bc --load "mytravel.xml"`
+
+Set lft path to be used by BC traces:
+
+`$ python bc -l "/usr/bin/lft"`
+
+
+### License
+
+Border Check is free software, and may be redistributed under [GPL v3].
+
+
+### Contribute: 
+
+If you want to contribute to BC development, reporting a bug, providing a patch, commenting 
+on the code base or simply need to find help to run it, please go to:
+
+      irc.freenode.net / channel: #BorderCheck
+
+If nobody gets back to you, then drop an e-mail.
+
+To make donations use the following hashes:
+  
+      - Bitcoin: 1Q63KtiLGzXiYA8XkWFPnWo7nKPWFr3nrc
+      - Ecoin: ETtSteMWxjY7RKWZGMNSkX7eC3BJ21VYXE
+
+
+### Contact
+
+Please report any problems you encounter using/installing Border Check to:
+
+ - Roel Roscam Abbing (rscmbbng@riseup.net)
+ - psy (epsylon@riseup.net)
 
-Border Check (BC) is a browser extension that illustrates the physical and political realities of the internet's infrastructure using free software tools.

bc

@@ -0,0 +1,20 @@
+#!/usr/bin/env python2
+# -*- coding: iso-8859-15 -*-
+"""
+BC (Border-Check) is a tool to retrieve info of traceroute tests over website navigation routes.
+GPLv3 - 2013-2014-2015 by psy (epsylon@riseup.net)
+"""
+from main import bc
+
+class NullOutput(object):
+    def write(self, text):
+        pass
+    def flush(self):
+        pass
+
+if __name__ == "__main__":
+    app = bc()
+    options = app.create_options()
+    if options:
+        app.set_options(options)
+        app.run()

doc/AUTHORS

@@ -0,0 +1,2 @@
+psy (epsylon@riseup.net)
+Roel Roscam Abbing (rscmbbng@riseup.net)

doc/CHANGELOG

@@ -0,0 +1,33 @@
+================================================================
+Changelog: Border-Check
+==============================
+
+=================
+May 13, 2015:
+=================
+
+- added "wizard" installer
+- added LFT binary to source
+- added more maps
+- added metadata
+- added travelling feature on map
+- added history of urls visited on map
+- added controllers to map (play|hide|stop)
+- fixed some minor bugs
+- public release: Border Check v0.2
+
+=================
+October 4, 2013:
+=================
+
+- added feature to import traces from XML template
+- fixed some LFT subprocess errors on Unix based systems
+- more clean non-debug mode
+- added 'localhost' filter to target parser
+
+=================
+September 27, 2013:
+=================
+
+- public release: Border Check v0.1
+

doc/INSTALL

@@ -0,0 +1,94 @@
+========================
+Border Check v0.2 - 2015
+========================
+
+Border Check (BC) is a browser extension that illustrates the physical and political realities of the internet's infrastructure using free software tools.
+
+As one surfs the net, data packets are sent from the user's computer to the target server. These data packets go on a journey hopping from server to server, potentially crossing multiple countries, until the packets reach the desired website. In each of the countries that are passed different laws and practices can apply to the data, influencing whether or not authorities can inspect, store or modify that data.
+
+In realtime BC lets you know which countries you surf through as you browse the web. Additionally BC will illustrate this process on a world map and (where available) provide you with contextualizing information on that country's laws and practices regarding your data.
+
+==========================
+INSTALL:
+===================
+
+BC runs on OSx and Unix systems. It requires the following libraries/dependencies:
+
+===================
+    + Python:
+===================
+
+        - python-geoip - Python bindings for the GeoIP IP-to-country resolver library
+
+               https://pypi.python.org/pypi/pygeoip/0.2.7
+
+        - python-lxml - pythonic binding for the libxml2 and libxslt libraries
+
+               https://pypi.python.org/pypi/lxml/3.2.3
+
+                       * On Debian-based systems (ex: Ubuntu), first run: 
+
+                              sudo apt-get install python-pip 
+
+                              and then run:
+      
+                                   pip install lxml
+           
+                                   pip install pygeoip
+        - biplist:
+
+               https://pypi.python.org/pypi/biplist/0.5
+
+===================
+    + Javascript:
+===================
+
+        - leaflet.js
+
+               http://leafletjs.com
+
+        - leaflet.markercluster.js
+
+               https://github.com/Leaflet/Leaflet.markercluster
+
+        - jquery.js
+
+               http://www.jquery.com
+
+        - rlayer.js
+
+               http://dynmeth.github.io/RaphaelLayer/
+
+        - raphael.js
+
+               http://raphaeljs.com/
+
+        - favicon.js
+
+               https://gist.github.com/mathiasbynens/428626
+
+===================
+    + LFT:
+===================
+
+        - LFT 3.35 (tracerouter)
+
+               http://pwhois.org/lft/
+
+===================
+    + Databases:
+===================
+
+        - Maxmind GeoIPLite and GeoIPASNum:
+          
+               http://dev.maxmind.com/geoip/legacy/geolite/
+
+===================
+
+Please report any problems you encounter using/installing Border Check to:
+
+ - Roel Roscam Abbing (rscmbbng@riseup.net)
+ - psy (epsylon@riseup.net)
+
+=========
+

doc/README

@@ -0,0 +1,96 @@
+Border Check v0.2 - 2015
+========================
+
+Border Check (BC) is a browser extension that illustrates the physical and political realities of the internet's infrastructure using free software tools.
+
+As one surfs the net, data packets are sent from the user's computer to the target server. These data packets go on a journey hopping from server to server, potentially crossing multiple countries, until the packets reach the desired website. In each of the countries that are passed different laws and practices can apply to the data, influencing whether or not authorities can inspect, store or modify that data.
+
+In realtime BC lets you know which countries you surf through as you browse the web. Additionally BC will illustrate this process on a world map and (where available) provide you with contextualizing information on that country's laws and practices regarding your data.
+
+Currently supporting the following browsers on OSX and Unix systems: Firefox, Chromium, Chrome, Safari
+
+NOTE: Browser history has to be enabled.
+
+See the included examples for usage.
+
+```
+bc [OPTIONS] 
+
+  --version             show program's version number and exit
+  -h, --help            show this help message and exit
+  -d, --debug           debug mode
+  -l LFT_PATH           path to lft (fetch from source or use provided binary)
+  --xml=EXPORT_XML      export traces to xml (ex: --xml foo.xml)
+  --load=IMPORT_XML     import traces (non root required) (ex: --load bar.xml)
+  --bh=BROWSER_HISTORY  set browser's history path
+  -b BROWSER            set browser type to be used: F = Firefox / C = Chrome / S = Safari / Ch = Chromium / N = None
+
+```
+
+#### Examples of usage:
+
+Normal BC run:
+
+`$ python bc`
+
+Debug mode:
+
+`$ python bc --debug`
+
+Export 'tracing' results to xml:
+
+`$ python bc --xml "mytravel.xml"`
+
+
+#### More options (set browsers, paths, etc):
+
+Use Firefox browser: 
+
+`$ python bc -b F`
+
+Use Chromium browser: 
+
+`$ python bc -b Ch` 
+
+Set browser's history path manually, on Galeon/Unix:
+
+`$ python bc --bh ".galeon/mozilla/galeon/places.sqlite"`
+
+Set browser's history path manually, on Chrome/OSx:
+
+`$ python bc --bh "Library/Application Support/Google/Chrome/Default/History"`
+
+Set browser's history path manually, on Safari/OSx:
+
+`$ python bc --bh "Library/Safari/History.plist"` 
+
+Set browser's history path manually, on Firefox/Unix:
+
+`$ python bc --bh "Library/Safari/History.plist"` 
+
+Import 'traces' from xml:
+
+`$ python bc --load "mytravel.xml"`
+
+Set lft path to be used by BC traces:
+
+`$ python bc -l "/usr/bin/lft"`
+
+
+### Contribute: 
+
+If you want to contribute to BC development, reporting a bug, providing a patch, commenting 
+on the code base or simply need to find help to run it, please go to:
+
+      irc.freenode.net / channel: #BorderCheck
+
+If nobody gets back to you, then drop an e-mail:
+
+    - Roel Roscam Abbing (rscmbbng@riseup.net)
+    - psy (epsylon@riseup.net)
+
+To make donations use the following hashes:
+  
+      - Bitcoin: 1Q63KtiLGzXiYA8XkWFPnWo7nKPWFr3nrc
+      - Ecoin: ETtSteMWxjY7RKWZGMNSkX7eC3BJ21VYXE
+

doc/THANKS

@@ -0,0 +1 @@
+UFO & Ikujam

options.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python
+# -*- coding: iso-8859-15 -*-
+"""
+BC (Border-Check) is a tool to retrieve info of traceroute tests over website navigation routes.
+GPLv3 - 2013-2014-2015 by psy (epsylon@riseup.net)
+"""
+import optparse
+import json
+
+class BCOptions(optparse.OptionParser):
+    def __init__(self, *args):
+        optparse.OptionParser.__init__(self, 
+                           prog='bc.py',
+			   version='\nBC (Border-Check) v0.2 - 2015 - (GPLv3.0)\n',
+                           usage= '\n\nbc [OPTIONS]')
+ 
+        self.add_option("-w", "--wizard", action="store_true", dest="wizard", help="wizard installer")
+        self.add_option("-d", "--debug", action="store_true", dest="debug", help="debug mode")
+        self.add_option("-l", action="store", dest="lft_path", help="path to lft (fetch from source or use provided binary)")
+        self.add_option("--xml", action="store", dest="export_xml", help="export traces to xml (ex: --xml foo.xml)")
+        self.add_option("--load", action="store", dest="import_xml", help="import traces (non root required) (ex: --load bar.xml)")
+        self.add_option("--bh", action="store", dest="browser_history", help="set browser's history path")
+        self.add_option("-b", action="store", dest="browser", help="set browser type to be used: F = Firefox / C = Chrome / S = Safari / Ch = Chromium / N = None")
+        #self.add_option("--proxy", action="store", dest="proxy", help="set proxy server")
+        self._options={}
+        
+    def get_options(self, user_args=None):
+        (options, args) = self.parse_args(user_args)
+        self._options=options
+        return options
+
+
+    def save_options(self):
+        optionfile=open("options.json","w")
+        json.dump(self._options,optionfile)

web/ajax.py

@@ -0,0 +1,116 @@
+#!/usr/bin/env python
+# -*- coding: iso-8859-15 -*-
+"""
+BC (Border-Check) is a tool to retrieve info of traceroute tests over website navigation routes.
+GPLv3 - 2013-2014-2015 by psy (epsylon@riseup.net)
+"""
+from xml.dom.minidom import parseString
+import xml.etree.ElementTree as ET
+import re
+import traceback
+reload=False
+
+try:
+  bc_status_file = open('bc.status')
+  s = str(bc_status_file.read())
+  if s =='fresh':
+    with open('bc.status', 'w') as file:
+      file.write("old")
+    reload=True
+except:
+  pass
+
+# todo :
+# * call ajax.py from here
+# * save last url timestamp as get parameter
+# * do parsing only if timestamp changes
+# * ui integration
+
+#function to split ISP company names from ASN
+def ASN_Split(asn):
+  name_parts = []
+  for i in asn.split():
+    if re.match(r'AS\d{1,6}$', i):
+      asn = i
+    elif not re.match(r'AS\d{1,6}$', i):
+      name_parts.append(i)
+  company = ' '.join(name_parts)
+  return (asn, company)
+
+
+def extract_data():
+  url=""
+  hop_list = []
+  hop_ip_list =[]
+  geoarray = []
+  latlong= []
+  asn_list =[]
+  server_name_list = []
+  timestamp_list = []
+  telco_list = []
+  country_code_list = []
+  last_hop = 0
+  n_hops=0
+  unique_country_code_list = []
+  # extract data from a xml file
+  f = open('data.xml', 'r')
+  f2 = open('data.xml', 'r')
+  xml = ET.parse(f)
+  data = f2.read()
+  dom = parseString(data.encode('utf-8'))
+  f.close()
+  f2.close()
+  last_hop = int(xml.findall('hop')[-1].text)
+  n_hops = dom.getElementsByTagName('hop')[-1].toxml().replace('<hop>', '').replace('</hop','')
+
+
+  for counter in range(0, last_hop+1):
+    url = xml.getroot().text
+    hop_element = parseString(dom.getElementsByTagName('hop')[counter].toxml().encode('utf-8'))
+    hop = xml.findall('hop')[counter].text
+    server_name = hop_element.getElementsByTagName('server_name')[0].toxml().replace('<server_name>','').replace('</server_name>','')
+    asn = hop_element.getElementsByTagName('asn')[0].toxml().replace('<asn>','').replace('</asn>','')
+    hop_ip = hop_element.getElementsByTagName('hop_ip')[0].toxml().replace('<hop_ip>','').replace('</hop_ip>','')
+    longitude = hop_element.getElementsByTagName('longitude')[0].toxml().replace('<longitude>','').replace('</longitude>','')
+    latitude = hop_element.getElementsByTagName('latitude')[0].toxml().replace('<latitude>','').replace('</latitude>','')
+    timestamp = hop_element.getElementsByTagName('timestamp')[0].toxml().replace('<timestamp>','').replace('</timestamp>','')
+    country_code = hop_element.getElementsByTagName('country_code')[0].toxml().replace('<country_code>','').replace('</country_code>','')
+
+    latlong = [float(latitude.encode('utf-8')), float(longitude.encode('utf-8'))]
+    geoarray.append(latlong)
+    asn_list.append(ASN_Split(asn.encode('utf-8'))[0])
+    telco_list.append(ASN_Split(asn.encode('utf-8'))[1])
+    hop = int(hop) +1
+    hop_list.append(str(hop))
+    hop_ip_list.append(hop_ip.encode('utf-8'))
+    server_name_list.append(server_name.encode('utf-8'))
+    timestamp_list.append(float(timestamp))
+    country_code_list.append(country_code.encode('utf-8'))
+
+    unique_country_code_list = set(country_code_list)
+  return """
+<script type="text/javascript">
+    counter_max = """+str(last_hop)+"""
+    latlong = """+str(geoarray)+"""
+    asn_list = """+str(asn_list)+"""
+    hop_ip_list = """+str(hop_ip_list)+"""
+    telco_list = """+str(telco_list)+"""
+    server_name_list = """+str(server_name_list)+"""
+    timestamp_list = """+str(timestamp_list)+"""
+    country_code_list = """+str(country_code_list)+"""
+    unique_country_code_list = """+str(list(unique_country_code_list))+"""
+    bcHistory.add('"""+url+"""',Array(counter_max,latlong,asn_list, hop_ip_list,telco_list,server_name_list,timestamp_list,country_code_list,unique_country_code_list))
+</script>
+"""
+
+try:
+  if reload:
+    output=extract_data()
+
+except:
+  args['error']="No data available"
+  traceback.print_exc()
+  reload=False
+
+if 'error' in args:
+  output=output+'<script>$("#error").html("'+args['error']+'")</script>'