Last update on tue mar 26 16:25 UTC 2012
The Web Crawler Security is a python based tool to automatically crawl a web site. It is a web crawler oriented to help in penetration testing tasks. The main task of this tool is to search and list all the links (pages and files) in a web site.
The crawler has been completely rewritten in v1.0 bringing a lot of improvements: improved the data visualization, interactive option to download files, increased speed in crawling, exports list of found files into a separated file (useful to crawl a site once, then download files and analyse them with FOCA), generate an output log in Common Log Format (CLF), manage basic authentication and more!
Many of the old features has been reimplemented and the most interesting one is the capability of the crawler to search for directory indexing.
Feature
- Crawl http and https web sites (even web sites not using common ports). Crawl http and https web sites (even web sites not using common ports). Crawl http and https web sites (even web sites not using common ports).
- (new!) It allows to determine the depth of the crawling (-C <depth> option)
- (new!) Generates a summary at the end of the crawling with statistics about the crawl results
- (new!) Implemented HEAD method for analysing file types before crawling. This feature improves the speed of the crawler significantly.
- Uses regular expressions to find 'href', 'src' and 'content' links.
- Identifies relative links.
- Identifies non-html files and shows them.
- Not crawl non-html files.
- Identifies directory indexing.
- Crawl directories with indexing (not yet implemented in v1.0)
- Uses CTRL-C to stop current crawler stages and continue working. Very useful stuff...
- Identifies all kind of files by reading the content-type header field of the response.
- Exports (-e option) in a separated file a list of all files URLs found during crawling.
- Select type of files to download (-d option). Ex.: png,pdf,jpeg,gif or png,jpeg.
- Select in an interactive way which type of files to download (-i option).
- Save the downloaded files into a directory. It only creates the output directory if there is at least one file to download.
- Generates a output log in CLF (Common Log Format) of all the request done during crawling.
- (beta) Login with basic authentication. Feedback is welcome!
- Tries to detect if the website uses a CMS (like wordpress, joomla, etc) (not yet implemented in v1.0)
- It looks for '.bk' or '.bak' files of php, asp, aspx, jps pages. (not yet implemented in v1.0)
- It identifies and calculates the number of unique web pages crawled. (not yet implemented in v1.0)
- It identifies and calculates the number of unique web pages crawled that contains parameters in URL. (not yet implemented in v1.0)
- It works in Windows, but didn't save results yet
Domain Analyzer Security Tool
Domain analyzer is a security analysis tool which automatically discovers and reports information about the given domain. Its main purpose is to analyze domains in an unattended way....
PyLogAnalyser
PyLogAnalyzer is a tool that receives an input log in black and white, a configuration INI file, which contains the list of rules to process the input, and an output file where to save the results.
These rules permit to detect an input line according to a regular expression (regex) or line number ra.........
JDynamiTe, Dynamic Template in Java
JDynamiTe is a tool which allows you to dynamically create documents in any format from "template" documents.
And very few lines of code (or no line at all!) are needed to do that.
Some typical usage domains of JDynamiTe are:
- dynamic Web pages creation,
- text document generation,
- source c.........
Monastic University
This software will help in the management of Admissions / Withdrawals of Staffs & Students & also processing of Exam Marks of all Monastic Universities. Designed with goals of Simplicity & packaged with all necessary open source tools to get you into action, straightaway.
Just download the file gi.........
miRDeep*
Please cite: An, J., Lai, J., Lehman, M.L. and Nelson, C.C. (2013) miRDeep*: an integrated application tool for miRNA identification from RNA sequencing data. Nucleic Acids Res, 41, 727-737.
We will create index for you if you tell us your interested species (j.anATqut.edu.au).
download command l.........
Ryouko
This project has been moved to https://github.com/foxhead128/ryouko...
torotools: Social DMS HRMS Time Tracking
torotools.es is a software suite of responsive design web products for managing your company´s knowledge and talent. Based on Material Design.
toro ECM tool is a free web based Enterprise Content Management, designed to help your company to improve the creation and management of information. It is .........
Lioness (Languages Interop Framework)
Framework for making Windows applications that are one .exe file in AutoHotKey_L,C++,C#, VB.NET,Java,Groovy,Common Lisp,Nemerle,Ruby,Python,PHP,Lua,Tcl,Perl,Jint,S#,WSH VBScript,HTML/JavaScript/CSS,COM, PowerShell without compiling . For .NET 4....
bWAPP
bWAPP, or a buggy web application, is a free and open source deliberately insecure web application.
bWAPP helps security enthusiasts, developers and students to discover and to prevent web vulnerabilities. bWAPP prepares one to conduct successful penetration testing and ethical hacking projects. Wh.........
MentoLiga
Manages the Football!
Creates, Manages any type of leagues.
Manage multiple leagues.
Tracking the league round to round.
Results, standings, scorers, assists, disciplinary, substitutions, injuries, results table and betting.
Easy Data Entry System. (Automatic and Multi-Selection)
Own Goals Tracking.........
