wget
wget is a GNU utility for retrieving files over the web using the popular internet transfer protocols (HTTP, HTTPS, FTP). It's useful either for obtaining individual files or mirroring entire web sites, as it can convert absolute links in downloaded documents to relative links. The GNU wget manual is the definitive resource.
Is there a built-in command-line tool under Windows like wget/curl? Ask Question Asked 8 years, 1 month ago. Active 9 months ago. Viewed 232k times 47. I come from a Linux/Unix background and I have been wondering if Windows has a binary that can download files from the console. I would like to automate a certain process and one of my.
- Linux curl & wget. These commands are used to download a file from the internet using CLI. With curl commands option O will be used while wget command will be used without any option. The file will be saved in the current directory. Syntax: Example: Look at the above snapshot, we have downloaded one file with the help of curl -O command.
- Wget curl; Wget is a simple tool designed to perform quick downloads: Curl on the other hand is a much more powerful tool. It can achieve a lot more as compared to Wget. Wget is command line only: Curl is powered by libcurl: Wget supports only HTTP, HTTPS, and FTP protocols.
- Wget and cURL are two complementary internet utility. The first is known for site-mirroring/crawler and the second for downloading and uploading from various protocols.
Some Useful wget Switches and Options
Usage is wget option url1 url2
| Option | Purpose |
|---|---|
| -A, -R | Accept and reject lists; -A.jpg will download all .jpgs from remote directory |
| --backup-converted | When converting links in a file, back up the original version with .orig suffix; synonymous with -K |
| --backups=backups | Back up existing files with .1, .2, .3, etc. before overwriting them; the 'backups' directive specifies the maximum backups made for each file |
| -c | Continue an interrupted download |
| --convert links | Convert links in downloaded files to point to local files |
| -i file | Specify input file from which to read URLs |
| -l depth | Specify maximum recursion depth; default is 5 |
| -m | Shortcut for mirroring options: -r -N -l inf --no-remove-listing, i.e., turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings |
| -N | Turns on timestamping |
| -O file | Specify the name of an output file, if you want it to be different than the downloaded file |
| -p | Download prerequisite files for displaying a web page (.css, .js, images, etc.) |
| -r | Download files recursively [RTFM here as it can get ugly fast] |
| -S | Print HTTP headers or FTP responses from remote servers |
| -T seconds | Set a timeout for an operation that takes too long |
| --user=user --password=password | Specify the username and/or password for HTTP/FTP logins |
Some wget Examples
Basic file download:
Download a file, rename it locally:
Download multiple files:
Download a file with your HTTP or FTP login/pass encoded:
Retrieve a single web page and all its support files (css, images, etc.) and change the links to reference the downloaded files:
Retrieve the first three levels of tldp.org, saving them to local directory tldp:
Create a five levels-deep mirror of TLDP, keeping its directory structure, re-pointing the links to local files, saving the activity log to tldplog:
Download all JPEGs from a from a given web directory, but not its child or parent directories:
Mirror a site in the specified local directory, converting links for local viewing, backing up the original HTML files locally as *.orig before rewriting the links:
cURL
cURL is a free software utility for transferring data over a network. Although cURL can retrieve files over the web like wget, it speaks many more protocols (HTTP/S, FTP/S, SCP, LDAP, IMAP, POP, SMTP, SMB, Telnet, etc.), and it can both send and reliably read and interpret server commands. cURL can send HTTP POST headers to interact with HTML forms and buttons, for example, and if it receives a 3xx HTTP response (moved), cURL can follow the resource to its new location. Keep in mind that cURL thinks in terms of data streams, not necessarily in terms of tidy, human-readable files. The cURL manpage is the definitive resource.
Some cURL Examples
Get the index.html file from a web site, or save index.html locally with a specified file name, or save the file locally using its remote name:
Wget Curl Windows
FTP -- get a particular file, or a directory listing:
Download a file from a web site that uses a redirect script, like Sourceforge (-L tells cURL to observe the Location header):
Specify a port number:
Specify a username and password:
Get a file over SSH (scp) using an RSA key for password-less login:
Get a file from a Samba server:
Send an email using Gmail:
Get a file using an HTTP proxy that requires login:
Get the first or last 500 bytes of a file:
Upload a file to an FTP server:
Show the HTTP headers returned by a web server, or save them to a file:
Send POST data to an HTTP server:
Emulate a fill-in form, using local file myfile.txt as the source for the 'file' field:
Set a custom referrer or user agent:
Categories:
I participate in the Amazon Associates program. Making your Amazon purchases through my affiliate links supports this site at no additional cost to you:
bittorrent vs HTTP, curl vs libcurl and curl vs HTTPie.The main differences as I (Daniel Stenberg) see them. Please consider my bias towards curl since after all, curl is my baby - but I contribute to Wget as well.
Please let me know if you have other thoughts or comments on this document.
File issues or pull-requests if you find problems or have improvements.
What both commands do
- both are command line tools that can download contents from FTP, HTTP(S)
- both can send HTTP POST requests
- both support HTTP cookies
- both support metalink, HSTS and HTTP proxy
- both are designed to work without user interaction
- both are fully open source and free software
- both projects started in 1996 (under other names)
- both are portable and run on many operating systems
curl
library: curl is powered by libcurl - a cross-platform library with a stable API that can be used by each and everyone. This difference is major since it creates a completely different attitude on how to do things internally. It is also slightly harder to make a library than a 'mere' command line tool.
pipes. curl works more like the traditional Unix cat command, it sends more stuff to stdout, and reads more from stdin in a 'everything is a pipe' manner. Wget is more like cp, using the same analogue.
Single shot: curl is basically made to do single-shot transfers of data. It transfers just the URLs that the user specifies, and does not contain any recursive downloading logic nor any sort of HTML parser.
More protocols: curl supports FTP(S), GOPHER(S), HTTP(S), SCP, SFTP, TFTP, TELNET, DICT, LDAP(S), MQTT, FILE, POP3(S), IMAP(S), SMB(S), SMTP(S), RTMP and RTSP. Wget supports HTTP(S) and FTP.
More portable: curl builds and runs on lots of more platforms than wget. For example: OS/400, TPF and other more 'exotic' platforms that aren't straight-forward Unix clones. curl requires but a C89 compiler.
More SSL libraries and SSL support: curl can be built with one out of thirteen (13!) different SSL/TLS libraries, and it offers more control and wider support for protocol details.
HTTP auth: curl supports more HTTP authentication methods, especially over HTTP proxies: Basic, Digest, NTLM and Negotiate
SOCKS: curl supports SOCKS4 and SOCKS5 for proxy access. With local or proxy based name resolving.
Bidirectional: curl offers upload and sending capabilities. Wget only offers plain HTTP POST support.
HTTP multipart/form-data sending, which allows users to do HTTP 'upload' and in general emulate browsers and do HTTP automation to a wider extent.
curl supports gzip, brotli, zstd and deflate Content-Encoding and does automatic decompression.
curl offers and performs decompression of Transfer-Encoded HTTP, wget doesn't.
curl supports HTTP/2, HTTP/3, alt-svc and it does dual-stack connects using Happy Eyeballs.
curl can do many transfers in parallel (
-Z).Much more developer activity. While this can be debated, I consider three metrics here: mailing list activity, source code commit frequency and release frequency. Anyone following these two projects can see that the curl project has a lot higher pace in all these areas, and it has been so for 15+ years. Compare on openhub.
curl comes pre-installed on macOS and Windows 10. Wget does not.
Wget
Wget is command line only. There's no library.
Recursive!: Wget's major strong side compared to curl is its ability to download recursively, or even just download everything that is referred to from a remote resource, be it a HTML page or a FTP directory listing.
Older: Wget has traces back to its predecessor from January 9, 1996, while curl can be tracked back no earlier than to November 11, 1996.
GPL: Wget is GPL v3. curl is MIT licensed.
GNU: Wget is part of the GNU project and all copyrights are assigned to FSF. The curl project is entirely stand-alone and independent with no organization parenting at all with almost all copyrights owned by Daniel.
Wget requires no extra options to simply download a remote URL to a local file, while curl requires -o or -O.
Wget supports only GnuTLS or OpenSSL for SSL/TLS support.
Wget supports only Basic auth as the only auth type over HTTP proxy.
Wget has no SOCKS support.
Its ability to recover from a prematurely broken transfer and continue downloading has no counterpart in curl.
Wget enables more features by default: cookies, redirect-following, time stamping from the remote resource etc. With curl most of those features need to be explicitly enabled.
There's a 'wget' in BusyBox, there's no curl there (it is not the actual wget, just a stripped down clone with the same name).
Wget can be typed in using only the left hand on a qwerty keyboard!
Wget requires a C99 compiler and also relies on gnulib.
Primarily: use the one that gets the job done for you.
Wget has (recursive) downloading powers that curl does not feature and it also handle download retries over unreliable connections possibly slightly more effective.
For just about everything else, curl is probably the more suitable tool.
In recent years, wget2 is worked on to become the replacement for wget. This comparison will eventually get wget2 details as well.
Two other capable tools with similar feature set include aria2 and axel - try them out!
What Curl Type Is My Hair
For a stricter feature by feature comparison (that also compares other similar tools), see the curl comparison table
Wget Curl Install
Feedback and improvements by: Micah Cowan, Olemis Lang
Wget Curl Binary
Updated: March 22, 2021 09:23 (Central European, Stockholm Sweden)