View Issue Details

IDProjectCategoryView StatusLast Update
0001382Kali Linux[All Projects] New Tool Requestspublic2014-06-09 18:25
ReporterkarkassaAssigned Todookie  
PrioritynormalSeverityminorReproducibilityhave not tried
Status resolvedResolutionfixed 
Product Version 
Target VersionFixed in Version1.0.8 
Summary0001382: Parsero: The tool to audit the Robots.txt automatically
DescriptionParsero is a free script written in Python which read the Robots.txt file of a web server and look at the Disallow entries in order to connect with them. Parsero prints if they are available or not. This tool is really faster and help you out when you are auditing a website trying to locate files or directories on a web server which have been hidden to the search engines by the web administrators. I haven't found any similar tool to achieve this goal.

I would be really happy if you consider to try this tool.

You can get the latest version here:

https://github.com/behindthefirewalls/Parsero

More info:

http://www.behindthefirewalls.com/2013/12/parsero-tool-to-audit-robotstxt.html

Please, let my know a feedback and if you want a real approach with real examples.
Additional InformationPython3 and Urllib3 is needed.

Activities

2014-02-10 12:39

 

Parsero_v06_Kali_1.PNG (120,996 bytes)
Parsero_v06_Kali_1.PNG (120,996 bytes)

2014-02-10 12:39

 

Parsero_v06_Kali_2.PNG (280,760 bytes)

jnieto

2014-02-10 12:41

reporter   ~0002035

That the administrator write a robots.txt, it doesn't mean that the files or direcotories typed in this file will not be indexed by Bing, Google, Yahoo..

Now Parsero is capable of searching in Bing to locate content indexed whithout the web administrator authorization!!!

jnieto

2014-05-09 08:42

reporter   ~0002036

Update: Some times the links indexed by Bing are available and sometimes not, now Parsero check each one (in the first Bing results page) in order to print out if they are available or not.

2014-05-09 08:43

 

Parsero_v071.png (215,788 bytes)
Parsero_v071.png (215,788 bytes)

dookie

2014-06-09 18:25

administrator   ~0002072

Added in parsero_0.75-0kali1. It will be in the repos soon.

Thanks for the suggestion.

Issue History

Date Modified Username Field Change
2014-06-04 15:19 karkassa New Issue
2014-06-04 15:19 karkassa Issue generated from: 0000876
2014-06-09 18:00 dookie Assigned To => dookie
2014-06-09 18:00 dookie Status new => assigned
2014-06-09 18:25 dookie Note Added: 0002072
2014-06-09 18:25 dookie Status assigned => resolved
2014-06-09 18:25 dookie Fixed in Version => 1.0.8
2014-06-09 18:25 dookie Resolution open => fixed
2014-06-29 19:52 vichet Issue cloned: 0001478
2014-07-29 16:43 aniketbhole Issue cloned: 0001625