regex - Use wget to crawl specific URLs -

-

i trying crawl links website use download manager download files.

i've tried:

wget --wait=20 --limit-rate=20k -r -p -u mozilla "www.mywebsite.com"

i can't figure out how use wget or regular expressions save desired links only!

wget offers wide variety of options fine tuning files download in recursive crawl.

here few options can interest you:

  • --accept-regex urlregex

download url matching urlregex. urlregex regular expression matched against complete url.

  • --reject-regex urlregex

ignore url matching urlregex. urlregex regular expression matched against complete url.

  • -l

tells wget follow relative links.

relative links example:

<a href="foo.gif"> <a href="foo/bar.gif"> <a href="../foo/bar.gif">

non relative links:

<a href="/foo.gif"> <a href="/foo/bar.gif"> <a href="http://www.server.com/foo/bar.gif">

references


Comments

Post a Comment

Popular posts from this blog

Ansible - ERROR! the field 'hosts' is required but was not set -

-
i've error when launch playbook don't found why.... error! field 'hosts' required not set there main.yml : --- - hosts: hosts - vars: - elasticsearch_java_home: /usr/lib/jmv/jre-1.7.0 - elasticsearch_http_port: 8443 - tasks: - include: tasks/main.yml - handlers: - include: handlers/main.yml and /etc/ansible/hosts : [hosts] 10.23.108.182 10.23.108.183 10.23.108.184 10.23.108.185 when test ping, : [root@poste08-08-00 elasticsearch]# ansible hosts -m ping 10.23.108.183 | success => { "changed": false, "ping": "pong" } 10.23.108.182 | success => { "changed": false, "ping": "pong" } 10.23.108.185 | success => { "changed": false, "ping": "pong" } 10.23.108.184 | success => { "changed": false, "ping": "pong" } please, me :) regards, you have syntax error in playbook.
Read more

customize file_field button ruby on rails -

-
i'm trying rid of ugly button rails has default file upload. want add glyphicon <%= f.file_field :image_url, class:"glyphicon glyphicon-camera" %> this didn't work , have tried other things in different post saw on page, don't attach file. as suggested here bootstrap filestyle , can use bootstrap's filestyle style file upload buttons. step-1: add layout application.html.erb i have added bootstrap filestyle library cdn. should make sure have both jquery , boostrap loaded. <script type="text/javascript" src="https://cdn.jsdelivr.net/bootstrap.filestyle/1.1.0/js/bootstrap-filestyle.min.js"> </script> step-2: add corresponding js file: $(":file").filestyle({input: false}); step-3: image_url file field follows: <%= f.file_field :image_url,class: "filestyle", "data-input" => false %>
Read more

SoapUI on windows 10 - high DPI/4K scaling issue -

-
Image
soapui doesn't seem dpi-aware , displays small on high dpi screen (tiny text , buttons). other applications running fine (screen resolution 3840 x 2160). version : soapui 5.1.2 os : windows 10 i have tried: configure soapui run "disable display scaling on high dpi settings" - parts of soapui looking bigger , don't display (image) changing resolution changing font size (preferences > editor settings > select font...) therefore assume, soapui pretends dpi-aware, not scale itself. have same issue? this workaround until developers round making version dpi-aware. step 1: add registry key hkey_local_machine\software\microsoft\windows\currentversion\sidebyside\preferexternalmanifest (dword) 1 step 2: add manifest file 'soapui-5.2.1.exe.manifest' in same directory 'soapui-5.2.1.exe' content of manifest file: <?xml version="1.0" encoding="utf-8" standalone="yes"?> <assembly xml
Read more