Sichere Webserver(konfiguration)

Mozilla Observatory

Lokale Installation

• how to install observatory (tested on Ubuntu 16.04 LTS)

sudo apt-get install -y git libpq-dev postgresql redis-server python3 python3-pip

cd /opt/

sudo git clone https://github.com/mozilla/http-observatory.git

sudo su - postgres

createdb http_observatory

psql http_observatory < /opt/http-observatory/httpobs/database/schema.sql

psql http_observatory

\password httpobsapi #passwort festlegen z.B. its

\password httpobsscanner #passwort festlegen z.B. its

• exit db (\q)

• exit psql user (exit)

sudo vi /etc/postgresql/9.5/main/postgresql.conf #set max_connections = 512, shared_buffers = 256MB

sudo service postgresql restart

sudo useradd -m httpobs

sudo su - httpobs

cd /opt/http-observatory

pip3 install .

pip3 install -r requirements.txt --upgrade

exit

• everything from here has to be done for every start - Starting from normal user

• start scanner

sudo install -m 750 -o httpobs -g httpobs -d /var/run/httpobs /var/log/httpobs

sudo su - httpobs

echo export HTTPOBS_API_URL="http://localhost:57001/api/v1" >> ~/.profile

cd /opt/http-observatory/

HTTPOBS_DATABASE_USER="httpobsscanner" HTTPOBS_DATABASE_PASS="its" /opt/http-observatory/httpobs/scripts/httpobs-scan-worker

• open new Terminal to start api

sudo su - httpobs

cd /opt/http-observatory/

HTTPOBS_DATABASE_USER="httpobsapi" HTTPOBS_DATABASE_PASS="its" uwsgi --http :57001 --wsgi-file /opt/http-observatory/httpobs/website/main.py --processes 8 --callable app --master

Verwendung der API

• Aufruf zum Scannen einer neuene Seite starten
  • POST-Request
    
  • http://localhost:57001/api/v1/analyze?host=www.testseite.de
    
    • hidden=true&rescan=true
      

• Status eines Scans anzeigen
  • GET-Request
    
  • http://localhost:57001/api/v1/analyze?host=www.testseite.de
    

 {  
 "end_time": "Tue, 22 Mar 2016 21:51:41 GMT",
 "grade": "A",
 "response_headers": { ... },
 "scan_id": 1,
 "score": 90,
 "start_time": "Tue, 22 Mar 2016 21:51:40 GMT",
 "state": "FINISHED",
 "tests_failed": 2,
 "tests_passed": 9,
 "tests_quantity": 11
 }

• Resultate eines bestimmten Scans anzeigen (Scan-Nr.)
  
  • GET-Request
    
  • http://localhost:57001/api/v1/getScanResults?scan=1
    
  • Beispiel siehe unten
    

• Liste Letzter Scans anzeigen
  
  • GET-Request
    
  • http://localhost:57001/api/v1/getRecentScans?max=20
    
    • mit Option von bestimmten "Noten" in diesem Fall alle F, also 20 Punkte und weniger
      

 {
 "site1.mozilla.org": "A",
 "site2.mozilla.org": "B-",
 "site3.mozilla.org": "C+",
 "site4.mozilla.org": "F",
 "site5.mozilla.org": "F",
 "site6.mozilla.org": "E",
 "site7.mozilla.org": "F",
 "site8.mozilla.org": "B+",
 "site9.mozilla.org": "A+",
 "site0.mozilla.org": "A-"
 }

• Liste der möglichen Noten mit Aufteilung getesteter Seiten
  
  • GET-Request
    
  • http://localhost:57001/api/v1/getGradeDistribution

 {
 "A+": 3,
 "A": 6,
 "A-": 2,
 "B+": 8,
 "B": 76,
 "B-": 79,
 "C+": 80,
 "C": 88,
 "C-": 86,
 "D+": 60,
 "D": 110,
 "D-": 215,
 "E": 298,
 "F": 46770
 }

Beispiel für Resultat eines Scans

The tests object contains one test object for each test conducted by the HTTP Observatory. Each test object is contains the following values:

• expectation: the expectation for a test result going in
• name: the name of the test; this should be the same as the parent object's name
• output: artifacts related to the test; these can vary widely between tests and are not guaranteed to be stable over time.
  • data: generally as close to the raw output of the test as is possible. For example, in the strict-transport-security test, output -> data contains the raw Strict-Transport-Security header
  • ????: other values under output have keys that vary; for example, the strict-transport-security test has a includeSubDomains key that is either set to True or False. Similarly, the redirection test contains a route key that contains an array of the URLs that were redirected to. See example below for more available keys.
• pass: whether the test passed or failed; a test that meets or exceeds the expectation will be marked as passed
• result: result of the test
• score_description: short description describing what result means
• score_modifier: how much the result of the test affected the final score; should range between +5 and -50

 {
 "content-security-policy": {
   "expectation": "csp-implemented-with-no-unsafe",
   "name": "content-security-policy",
   "output": {
     "data": {
       "connect-src": [
         "'self'",
         "https://sentry.prod.mozaws.net"
       ],
       "default-src": [
         "'self'"
       ],
       "font-src": [
         "'self'",
         "https://addons.cdn.mozilla.net"
       ],
       "frame-src": [
         "'self'",
         "https://ic.paypal.com",
         "https://paypal.com",
         "https://www.google.com/recaptcha/",
         "https://www.paypal.com"
       ],
       "img-src": [
         "'self'",
         "data:",
         "blob:",
         "https://www.paypal.com",
         "https://ssl.google-analytics.com",
         "https://addons.cdn.mozilla.net",
         "https://static.addons.mozilla.net",
         "https://ssl.gstatic.com/",
         "https://sentry.prod.mozaws.net"
       ],
       "media-src": [
         "https://videos.cdn.mozilla.net"
       ],
       "object-src": [
         "'none'"
       ],
       "report-uri": [
         "/__cspreport__"
       ],
       "script-src": [
         "'self'",
         "https://addons.mozilla.org",
         "https://www.paypalobjects.com",
         "https://apis.google.com",
         "https://www.google.com/recaptcha/",
         "https://www.gstatic.com/recaptcha/",
         "https://ssl.google-analytics.com",
         "https://addons.cdn.mozilla.net"
       ],
       "style-src": [
         "'self'",
         "'unsafe-inline'",
         "https://addons.cdn.mozilla.net"
       ]
     }
   },
   "pass": false,
   "result": "csp-implemented-with-unsafe-inline-in-style-src-only",
   "score_description": "Content Security Policy (CSP) implemented with unsafe-inline inside style-src directive",
   "score_modifier": -5
 },
 "contribute": {
   "expectation": "contribute-json-with-required-keys",
   "name": "contribute",
   "output": {
     "data": {
       "bugs": {
         "list": "https://github.com/mozilla/addons-server/issues",
         "report": "https://github.com/mozilla/addons-server/issues/new"
       },
       "description": "Mozilla's official site for add-ons to Mozilla software, such as Firefox, Thunderbird, and SeaMonkey.",
       "name": "Olympia",
       "participate": {
         "docs": "http://addons-server.readthedocs.org/",
         "home": "https://wiki.mozilla.org/Add-ons/Contribute/AMO/Code",
         "irc": "irc://irc.mozilla.org/#amo",
         "irc-contacts": [
           "andym",
           "cgrebs",
           "kumar",
           "magopian",
           "mstriemer",
           "muffinresearch",
           "tofumatt"
         ]
       },
       "urls": {
         "dev": "https://addons-dev.allizom.org/",
         "prod": "https://addons.mozilla.org/",
         "stage": "https://addons.allizom.org/"
       }
     }
   },
   "pass": true,
   "result": "contribute-json-with-required-keys",
   "score_description": "Contribute.json implemented with the required contact information",
   "score_modifier": 0
 },
 "cookies": {
   "expectation": "cookies-secure-with-httponly-sessions",
   "name": "cookies",
   "output": {
     "data": {
       "sessionid": {
         "domain": ".addons.mozilla.org",
         "expires": null,
         "httponly": true,
         "max-age": null,
         "path": "/",
         "port": null,
         "secure": true
       }
     }
   },
   "pass": true,
   "result": "cookies-secure-with-httponly-sessions",
   "score_description": "All cookies use the Secure flag and all session cookies use the HttpOnly flag",
   "score_modifier": 0
 },
 "cross-origin-resource-sharing": {
   "expectation": "cross-origin-resource-sharing-not-implemented",
   "name": "cross-origin-resource-sharing",
   "output": {
     "data": {
       "acao": null,
       "clientaccesspolicy": null,
       "crossdomain": null
     }
   },
   "pass": true,
   "result": "cross-origin-resource-sharing-not-implemented",
   "score_description": "Content is not visible via cross-origin resource sharing (CORS) files or headers",
   "score_modifier": 0
 },
 "public-key-pinning": {
   "expectation": "hpkp-not-implemented",
   "name": "public-key-pinning",
   "output": {
     "data": null,
     "includeSubDomains": false,
     "max-age": null,
     "numPins": null,
     "preloaded": false
   },
   "pass": true,
   "result": "hpkp-not-implemented",
   "score_description": "HTTP Public Key Pinning (HPKP) header not implemented",
   "score_modifier": 0
 },
 "redirection": {
   "expectation": "redirection-to-https",
   "name": "redirection",
   "output": {
     "destination": "https://addons.mozilla.org/en-US/firefox/",
     "redirects": true,
     "route": [
       "http://addons.mozilla.org/",
       "https://addons.mozilla.org/",
       "https://addons.mozilla.org/en-US/firefox/"
     ],
     "status_code": 200
   },
   "pass": true,
   "result": "redirection-to-https",
   "score_description": "Initial redirection is to https on same host, final destination is https",
   "score_modifier": 0
 },
 "strict-transport-security": {
   "expectation": "hsts-implemented-max-age-at-least-six-months",
   "name": "strict-transport-security",
   "output": {
     "data": "max-age=31536000",
     "includeSubDomains": false,
     "max-age": 31536000,
     "preload": false,
     "preloaded": false
   },
   "pass": true,
   "result": "hsts-implemented-max-age-at-least-six-months",
   "score_description": "HTTP Strict Transport Security (HSTS) header set to a minimum of six months (15768000)",
   "score_modifier": 0
 },
 "subresource-integrity": {
   "expectation": "sri-implemented-and-external-scripts-loaded-securely",
   "name": "subresource-integrity",
   "output": {
     "data": {
       "https://addons.cdn.mozilla.net/static/js/impala-min.js?build=552decc-56eadb2f": {
         "crossorigin": null,
         "integrity": null
       },
       "https://addons.cdn.mozilla.net/static/js/preload-min.js?build=552decc-56eadb2f": {
         "crossorigin": null,
         "integrity": null
       }
     }
   },
   "pass": false,
   "result": "sri-not-implemented-but-external-scripts-loaded-securely",
   "score_description": "Subresource Integrity (SRI) not implemented, but all external scripts are loaded over https",
   "score_modifier": -5
 },
 "x-content-type-options": {
   "expectation": "x-content-type-options-nosniff",
   "name": "x-content-type-options",
   "output": {
     "data": "nosniff"
   },
   "pass": true,
   "result": "x-content-type-options-nosniff",
   "score_description": "X-Content-Type-Options header set to \"nosniff\"",
   "score_modifier": 0
 },
 "x-frame-options": {
   "expectation": "x-frame-options-sameorigin-or-deny",
   "name": "x-frame-options",
   "output": {
     "data": "DENY"
   },
   "pass": true,
   "result": "x-frame-options-sameorigin-or-deny",
   "score_description": "X-Frame-Options (XFO) header set to SAMEORIGIN or DENY",
   "score_modifier": 0
 },
 "x-xss-protection": {
   "expectation": "x-xss-protection-1-mode-block",
   "name": "x-xss-protection",
   "output": {
     "data": "1; mode=block"
   },
   "pass": true,
   "result": "x-xss-protection-enabled-mode-block",
   "score_description": "X-XSS-Protection header set to \"1; mode=block\"",
   "score_modifier": 0
 }
 }