No Description

maravento b7a9a03fe1 Update 02/08/2018 1 week ago
update b7a9a03fe1 Update 02/08/2018 1 week ago
LICENSE 70c0f794e5 Update 10/21/2016 1 year ago
README.md b7a9a03fe1 Update 02/08/2018 1 week ago
blackdomains.txt 70c0f794e5 Update 10/21/2016 1 year ago
blackweb.sh 25e44cf74f Update 02/07/2018 2 weeks ago
whitedomains.txt 70c0f794e5 Update 10/21/2016 1 year ago

README.md

Blackweb

License Version

Blackweb es un proyecto que pretende recopilar la mayor cantidad de listas negras públicas de dominios (para bloquear porno, descargas, drogas, malware, spyware, trackers, bots, redes sociales, warez, venta de armas, etc), con el objeto de unificarlas y hacerlas compatibles con Squid-Cache (Tested in v3.5.x). Para lograrlo, realizamos una depuración de urls, para evitar duplicados, dominios inválidos (validación de ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), y un filtrado con listas blancas de dominios (falsos positivos, como google, hotmail, yahoo, etc), para obtener una mega ACL, optimizada para Squid-Cache, libre de "overlapping domains" (e.g: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").

Blackweb is a project that aims to collect as many public domain blacklists (to block porn, downloads, drugs, malware, spyware, trackers, Bots, social networks, warez, arms sales, etc.), in order to unify them and make them compatible with Squid-Cache (Tested in v3.5.x ). To do this, we perform a debugging of urls, to avoid duplicates, invalid domains (validation, ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), and filter with white lists of domains (false positives such as google , hotmail, yahoo, etc.), to get a mega ACL, optimized for Squid-Cache, free of overlapping domains (eg: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").

Descripción / Description

File BL Domains File size
blackweb.txt 7.919.408 176,7 MB

Dependencias / Dependencies

git squid bash tar zip wget subversion python

Modo de uso / How to use

La ACL blackweb.txt ya viene optimizada para Squid-Cache. Descárguela con blackweb.sh. Por defecto, la ruta de blackweb.txt es /etc/acl y del script blackweb.sh es /etc/init.d / The ACL blackweb.txt is already optimized for Squid-Cache. Download it with blackweb.sh. By default, blackweb.txt path is /etc/acl and the script blackweb.sh is /etc/init.d

wget https://raw.githubusercontent.com/maravento/blackweb/master/blackweb.sh -O /etc/init.d/blackweb.sh
sudo chown root:root /etc/init.d/blackweb.sh && sudo chmod +x /etc/init.d/blackweb.sh
sudo /etc/init.d/blackweb.sh

Actualización BLs / Update BLs

También puede descargar el proyecto Blackweb y actualizar la ACL blackweb.txt en dependencia de sus necesidades / You can also download the Blackweb project and update the blackweb.txt ACL depending on your needs

git clone --depth=1 https://github.com/maravento/blackweb.git
sudo cp -f blackweb/update/bwupdate.sh /etc/init.d
sudo chown root:root /etc/init.d/bwupdate.sh
sudo chmod +x /etc/init.d/bwupdate.sh
sudo /etc/init.d/bwupdate.sh

El bash bwupdate.sh realiza captura, depuración y limpieza de dominios para evitar conflictos, sin embargo consume gran cantidad de recursos de hardware durante el procesamiento y puede tomar horas o días / The bash script bwupdate.sh performs capturing, debugging and domains cleanup to avoid conflicts, however it consumes a large amount of hardware resources during processing and can take hours or days

Verifique su ejecución / Check execution (/var/log/syslog):

Ejecución exitosa / Successful execution

Blackweb for Squid: Done 06/05/2017 15:47:14

Ejecución fallida / Execution failed

Blackweb for Squid: Abort 06/05/2017 15:47:14 Check Internet Connection

Regla de Squid-Cache / Squid-Cache Rule

Edit /etc/squid/squid.conf:

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
acl blackweb dstdomain -i "/etc/acl/blackweb.txt"
http_access deny blackweb

Edición / Edition

Blackweb contiene millones de dominios bloqueados, por tanto, editarla manualmente puede ser frustrante. Entonces, si detecta un falso positivo, utilice la ACL whitedomains.txt y reporte el incidente, para corregirlo en la próxima actualización. Lo mismo aplica para dominios no incluidos en Blackweb, que quiera bloquear, puede incluirlos en blackdomains / Blackweb contains million domains blocked therefore manually editing can be frustrating. Then, if it detects a false positive, use the ACL whitedomains.txt and report the incident to correct it in the next update. The same applies for domains not included in Blackweb, you want to block, you can include them in blackdomains

acl whitedomains dstdomain -i "/etc/acl/whitedomains.txt"
acl blackdomains dstdomain -i "/etc/acl/blackdomains.txt"
acl blackweb dstdomain -i "/etc/acl/blackweb.txt"
http_access allow whitedomains
http_access deny blackdomains 
http_access deny blackweb

blackdomains.txt contiene dominios no incluidos en Blackweb (e.g. .youtube.com .googlevideo.com, .ytimg.com) y whitedomains.txt contiene el subdominio accounts.youtube.com desde Feb 2014, Google utiliza el subdominio accounts.youtube.com para autenticar sus servicios / blackdomains.txt contains domains not included in Blackweb (e.g. .youtube.com .googlevideo.com, .ytimg.com) and whitedomains.txt contains subdomain accounts.youtube.com since February 2014, Google uses the accounts subdomain .youtube.com to authenticate their services.

Data Sheet (Sources - Repositories)

URLs Blacklists

Shallalist

Capitole - Direction du Système d'Information (DSI)

MESD blacklists

Yoyo Serverlist

Oleksiig Blacklist

HP Hosts-file

Winhelp2002

Cibercrime-Tracker

Joewein Blacklist

Tracking-Addresses

Adaway

Lehigh Malwaredomains

Easylist for adblockplus

Zeus tracker

Malwaredomain Hosts List

Malware-domains

malc0de

BambenekConsulting

openphish

Artists Against 419

Tracking Disconnect.me

Carl Spam

StevenBlack Hosts (Replaces Badd-Boyz-Hosts, Someonewhocares and KADhosts)

Ultimate Hosts Blacklist

Hacked Malware Web Sites

Nginx Ultimate Bad Bot Blocker

BlackURLs

UrlBlacklist (Server Down since July 2017)

Taz SpamDomains

Passwall SpamAssassin (Server Down since Dec 2016. Last Update included in BlackURLs. Mirror)

The Big List of Hacked Malware Web Sites

Web Miner

Anti-WebMiner

Ransomware

Ransomware Abuse

Debugging (URLs/TLDs Whitelists, Invalid Domains, etc)

TLDs IANA

TLDs Mozilla Public Suffix

Wikipedia Top Level Domains

WhiteURLs

RemoteURLs

ipv6-hosts (Partial)

O365IPAddresses (Partial)

University Domains and Names Data List

Invalid Domains/TLDs

Central Repo Dead Domains

Tools

Parse Domains (modified)

httpstatus

httpstatus bash

Debugging list

Contributions

Agradecemos a todos aquellos que han contribuido a este proyecto. Los interesados pueden contribuir, enviándonos enlaces de nuevas listas, para ser incluidas en este proyecto / We thank all those who have contributed to this project. Those interested can contribute, sending us links of new lists, to be included in this project

Licence

GPL-3.0

© 2018 Maravento Studio

Disclaimer

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.