Install and use Wget command like a pro
Spread the love

What is Wget?

Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

How to install Wget?

# Install wget in Ubuntu \ Debian Linux apt-get install wget # Install wget on RHEL / CentOS / Fedora yum install wget # Install wget on OpenSUSE zypper install wget # Install wget on ArchLinux pacman -Sy wget # Install wget on FreeBSD pkg install wget # Install wget Using FreeBSD Ports Collection portsnap fetch update cd /usr/ports/ftp/wget make install clean rehash ## or hash -r for 'bash/sh/ksh'
Code language: PHP (php)

How to use Wget?

  1. Now if we want do download a single file using HTTP we type:
# Download a single file wget http://site-name.com/file-name.tar.gz
Code language: Bash (bash)

2. You can download a web page and save it to a folder of your desire:

# Download a website wget -o index.html http://site-name.com/page-url # Download a website into a different folder wget --directory-prefix='./home/user/Downloads/site-name/' http://site-name.com/page-url
Code language: Bash (bash)

3. Here is how to download files from password protected websites:

# Download files from password protected websites wget ‐‐http-user=username ‐‐http-password=password http://site-name.com/path-secret/file.tar.gz
Code language: Bash (bash)

4. Another way is to download specific type of files from the website:

# Download specific type of files from the website # This will download all the mp3 files $ wget --level=2 --recursive --accept mp3 http://site-name.com # will download all jpeg files $ wget ‐‐level=1 ‐‐recursive ‐‐no-parent ‐‐accept jpg,JPG http://site-name.com/
Code language: Bash (bash)

5. A cool options is to download multiple files with different protocols:

# Download multiple files with different protocols wget http://site-name.com/file.tar.gz ftp://151.232.45.6/picture.jpg
Code language: Bash (bash)

6. It’s also possible to limit the bandwidth of a file you are downloading:

# Limit the bandwidth of a file you are downloading wget --limit-rate=50k http://site-name.com/file.rar
Code language: Bash (bash)

7. You can download a entire website if you wish with all the files and folders inside:

# Mirror entire websites (all its pages and assets) wget --mirror --no-parent --continue http://site-name.com
Code language: PHP (php)

8. You can put URLs in a file and then tell Wget to download all of the links in the file

# Download all of the links in the file wget ‐‐input filename.txt
Code language: PHP (php)

9. Here is how to resume a currently download file from where it was left

# Resume a currently download file from where it was left wget -c http://site-name.com/file.zip
Code language: PHP (php)

10. Download in the Background Using wget -b

# Download in the Background Using wget -b wget -b http://www.site-name.com/link/filename.tar.bz2
Code language: PHP (php)

11. Check the status of the download using tail -f

# Check the status of the download using tail -f tail -f wget-log Saving to: `filename.tar.bz2.4' 0K .......... .......... .......... .......... .......... 1% 65.5K 57s 50K .......... .......... .......... .......... .......... 2% 85.9K 49s 100K .......... .......... .......... .......... .......... 3% 83.3K 47s 150K .......... .......... .......... .......... .......... 5% 86.6K 45s 200K .......... .......... .......... .......... .......... 6% 33.9K 56s 250K .......... .......... .......... .......... .......... 7% 182M 46s 300K .......... .......... .......... .......... .......... 9% 57.9K 47s
Code language: PHP (php)

12. A nice trick is to test Download URL using Wget –spider

# Test download wget --spider http://site-name.com/link/file.tar.bz2
Code language: PHP (php)

13. And finally here is how to download only certain file types using wget -r -A

# Download only certain file types wget -r -A.pdf http://site-name.com/files-folder/
Code language: PHP (php)

In this lesson, we learned some of the coolest ways to handle Wget. A tool without which no system administrator can work. For additional examples you can refer to the man page of the program.

Here is a quick video install on ubuntu and download a entire web page

We hope you enjoyed this article. if that is so please rate this page with the stars bellow and subscribe to our YouTube channel.

Rate this post

Facebook Comments Box