Spread the love

What is Wget?

Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

How to install Wget?

# Install wget in Ubuntu \ Debian Linux apt-get install wget # Install wget on RHEL / CentOS / Fedora yum install wget # Install wget on OpenSUSE zypper install wget # Install wget on ArchLinux pacman -Sy wget # Install wget on FreeBSD pkg install wget # Install wget Using FreeBSD Ports Collection portsnap fetch update cd /usr/ports/ftp/wget make install clean rehash ## or hash -r for 'bash/sh/ksh'
Code language: PHP (php)

How to use Wget?

  1. Now if we want do download a single file using HTTP we type:
# Download a single file wget http://site-name.com/file-name.tar.gz
Code language: Bash (bash)

2. You can download a web page and save it to a folder of your desire:

# Download a website wget -o index.html http://site-name.com/page-url # Download a website into a different folder wget --directory-prefix='./home/user/Downloads/site-name/' http://site-name.com/page-url
Code language: Bash (bash)

3. Here is how to download files from password protected websites:

# Download files from password protected websites wget ‐‐http-user=username ‐‐http-password=password http://site-name.com/path-secret/file.tar.gz
Code language: Bash (bash)

4. Another way is to download specific type of files from the website:

# Download specific type of files from the website # This will download all the mp3 files $ wget --level=2 --recursive --accept mp3 http://site-name.com # will download all jpeg files $ wget ‐‐level=1 ‐‐recursive ‐‐no-parent ‐‐accept jpg,JPG http://site-name.com/
Code language: Bash (bash)

5. A cool options is to download multiple files with different protocols:

# Download multiple files with different protocols wget http://site-name.com/file.tar.gz ftp://151.232.45.6/picture.jpg
Code language: Bash (bash)

6. It’s also possible to limit the bandwidth of a file you are downloading:

# Limit the bandwidth of a file you are downloading wget --limit-rate=50k http://site-name.com/file.rar
Code language: Bash (bash)

7. You can download a entire website if you wish with all the files and folders inside:

# Mirror entire websites (all its pages and assets) wget --mirror --no-parent --continue http://site-name.com
Code language: PHP (php)

8. You can put URLs in a file and then tell Wget to download all of the links in the file

# Download all of the links in the file wget ‐‐input filename.txt
Code language: PHP (php)

9. Here is how to resume a currently download file from where it was left

# Resume a currently download file from where it was left wget -c http://site-name.com/file.zip
Code language: PHP (php)

10. Download in the Background Using wget -b

# Download in the Background Using wget -b wget -b http://www.site-name.com/link/filename.tar.bz2
Code language: PHP (php)

11. Check the status of the download using tail -f

# Check the status of the download using tail -f tail -f wget-log Saving to: `filename.tar.bz2.4' 0K .......... .......... .......... .......... .......... 1% 65.5K 57s 50K .......... .......... .......... .......... .......... 2% 85.9K 49s 100K .......... .......... .......... .......... .......... 3% 83.3K 47s 150K .......... .......... .......... .......... .......... 5% 86.6K 45s 200K .......... .......... .......... .......... .......... 6% 33.9K 56s 250K .......... .......... .......... .......... .......... 7% 182M 46s 300K .......... .......... .......... .......... .......... 9% 57.9K 47s
Code language: PHP (php)

12. A nice trick is to test Download URL using Wget –spider

# Test download wget --spider http://site-name.com/link/file.tar.bz2
Code language: PHP (php)

13. And finally here is how to download only certain file types using wget -r -A

# Download only certain file types wget -r -A.pdf http://site-name.com/files-folder/
Code language: PHP (php)

In this lesson, we learned some of the coolest ways to handle Wget. A tool without which no system administrator can work. For additional examples you can refer to the man page of the program.

Here is a quick video install on ubuntu and download a entire web page

We hope you enjoyed this article. if that is so please rate this page with the stars bellow and subscribe to our YouTube channel.

Facebook Comments Box