Introduction
With one of the numerous accessible protocols, for example, HTTP, HTTPS, FTP, FTP, and SFTP, users can send and receive data with servers employing Curl, a freely available open-source program. You can utilize Curl to create scripts for automating repetitive processes, for instance, because it is a command line tool. Curl command in Linux has a wide range of use cases it can handle.
Tools that support the required network protocols are required when sending data from or to a server. The most well-known programs for this kind of work on Linux are curl and wget.
You will learn how to execute the Linux curl command in the following sections and get a complete rundown of all the options.
What is the Curl Command?
Using several network protocols, data can get transferred using the command curl in Linux(which stands for “Client URL”). By indicating the appropriate URL along with the data that needs to be transmitted or received, it can communicate with a web and application server.
Libcurl, a mobile client-side URL transfer library, powers curl. It can be used in a script or straight on the command line. Curl is frequently used in the following situations:
- getting files off the internet
- testing at the endpoint
- Detecting and fixing errors
curl Syntax
Following is the fundamental curl syntax:
curl [options/URLs]Examples include
- curl https://www.gnu.org/gnu/gnu.html
When the curl command in Linux gets used, the system produces the HTML data retrieved at the supplied URL.
The file can get downloaded to your local machine using curl when you provide a URL that points to it:
curl [url] > [local-file]The download activity bar displays the number of files that have been downloaded thus far.
The protocol determines the syntax of URLs used as part of commands. When writing multiple URLs simultaneously that change in one place, braces get used:
- http://example.{first, second, third}.com
With brackets, the alphabetic series get written:
- ftp://ftp.url.com/file[1-100].txt
Because multiple patterns are permitted but nested sequences are not,
- http://url.com/archive[2010-2020]/vol[1-4]/part{a,b,c}.html
the curl protocol
For data transport, curl enables a wide range of protocols. The whole list is provided below.
| Protocol | Description |
| DICT | a network protocol for dictionaries that allows users to ask the servers about word definitions. |
| FILE | a URL pattern that curl can use to get a file from a local file system. |
| FTP, FTPS | File transport Protocol, which is used to move files across clients and servers. A second SSL/TLS security surface gets added to FTPS, a variation of the same protocol. |
| GOPHER, GOPHERS | A prior version of HTTP that was a legacy protocol for finding, distributing, as well as searching online documents. Using an additional SSL/TLS security layer, GOPHERS is a variation of the same protocol. |
| HTTP, HTTPS | Transferring data across the internet and the web requires the Hypertext Transfer Protocol. A second SSL/TLS security element gets added to HTTP in HTTPS, the protocol’s version. |
| IMAP, IMAPS | Accessing and managing email get done using the Internet Message Access Protocol. With an additional SSL/TLS security level, the protocol is now known as IMAPS. |
| LDAP, LDAPS | For the maintenance and access of dispersed directories, there is a protocol called Lightweight Directory Access Protocol. Adding an extra SSL/TLS security layer, the protocol is now known as LDAPS. |
| MQTT | Message Queuing Telemetry Transport is a protocol used to transfer data between tiny gadgets, typically Internet of Things platforms. |
| POP3, POP3S | Email retrieval from the server gets done via the Post Office Protocol version 3 protocol. Incorporating an additional SSL/TLS security layer, POP3S is a variation of the same protocol. |
| RTMP | Real-Time Messaging Protocol is a streaming platform for data, including audio and video. |
| RTSP | Management of streaming media servers gets done through the Real Time Streaming Protocol. |
| SCP | A mechanism for replicating files across an SSH server get called Secure Copy. |
| SFTP | SSH File Transfer Protocol is an SSH connection-based variant of the File Transfer Protocol. |
| SMB, SMBS | The Server Message Block protocol controls the shared usage of files and computer devices. Having an additional SSL/TLS security layer, the protocol is now known as SMBS. |
| SMTP, SMPTS | A mechanism for sending emails that makes it simple get called the Simple Mail Transfer mechanism. With an additional SSL/TLS security layer, SMTPS is a variation of the same protocol. |
| TELNET | is a bidirectional, interactive, text-based application layer protocol. |
| TFTP | For transferring file information to or from a remote site, utilize the trivial file transfer protocol. |
For your convenience, we have compiled a collection of 15 Linux curl commands below.
1. Check out the curl version
In addition to the curl Linux version, the -V or –version parameters will also provide the supporting protocols and features for the present version.
$ curl --version
2. Save a File
Curl’s -O and -o options can be used for downloading files. While the second option lets you choose a new filename and location, the first will save the file with the same name regardless of the remote location in the present working directory.
- $ curl -O http://yourdomain.com/yourfile.tar.gz # Save as yourfile.tar.gz
- $ curl -o newfile.tar.gz http://yourdomain.com/yourfile.tar.gz # Save as newfile.tar.gz
3. Continue a Download That Got Interrupted
It is fairly simple to restart a download that got stopped for whatever reason (for instance, by pressing Ctrl + c). When -C – (dash C, space dash) gets used, curl gets instructed to pick up where it left off with the download.
$ curl -C - -O http://yourdomain.com/yourfile.tar.gz4. Save several files
You can download info.html and about.html from http://yoursite.com or http://mysite.com simultaneously by issuing the command below.
$ curl -O http://yoursite.com/info.html -O http://mysite.com/about.html 
5. Access a File’s URLs
Downloading files via a collection of URLs in a file is possible when curl and xargs get used together.
$ xargs -n 1 curl -O < listurls.txt
6. Employ a proxy, either with or without authentication
If you can listen on port 8080 on proxy.yourdomain.com while being behind a proxy server, then you should.
- $ curl -x proxy.yourdomain.com:8080 -U user: password -O http://yourdomain.com/yourfile.tar.gz
When your proxy doesn’t need authentication, you can omit the -U user: password option.
7. HTTP Headers for Query
The remote web server can communicate more details about itself together with the query itself using HTTP headers. This gives the customer information about how the request is being processed.
You can do the following to get a website’s HTTP headers:
$ curl -I www.serverwala.comYour browser’s tools for developers also have access to this information.
8. Submit an HTTP POST request with the parameters
The firstName along with lastName parameters, in addition to their respective values, will get sent to https://yourdomain.com/info.php by the command that follows.
$ curl --data "firstName=John&lastName=Doe" https://yourdomain.com/info.phpThe behavior of a typical HTML form can be replicated using this advice.
9. Using authentication or not, download files via an FTP server
The command that follows will download your file.tar.gz in the present working directory when a remote FTP server anticipates connections at ftp://yourftpserver.
$ curl -u username:password -O ftp://yourftpserver/yourfile.tar.gzWhen the FTP server permits anonymous logins, you can omit the -u username:password command.
Also Read: Learn to use SCP Command in Linux
10. Is authentication required or not while uploading files to an FTP server
To use curl to upload a local file with the name mylocalfile.tar.gz to ftp://yourftpserver, follow these steps:
$ curl -u username:password -T mylocalfile.tar.gz ftp://yourftpserver11. Indicate the User Agent
The data supplied with an HTTP request includes the user agent. This identifies the browser that the client employed to submit the request. The default setting for our current version of curl will be shown below, and we’ll modify it to “I am a new web browser” afterward:
$ curl -I http://localhost –user-agent “I am a new web browser”
If you visit https://www.cnn.com, do you need to know which cookies get downloaded to your computer? To store them in cnncookies.txt, execute the following command. After that, you can view the file using the cat command.
$ curl --cookie-jar cnncookies.txt https://www.cnn.com/index.html -O
13. Transmit Website Cookies
The cookies you obtained in the previous step can be used in any other requests you make to the same website.
$ curl --cookie cnncookies.txt https://www.cnn.com14. Adjust the Name Resolution
When you’re a web developer and wish to test a local copy of yourdomain.com before publishing it, you can instruct curl to redirect http://www.yourdomain.com to your localhost as follows:
$ curl --resolve www.yourdomain.com:80:localhost http://www.yourdomain.com/As a result, curl will get instructed to use localhost to access the website rather than DNS or the /etc/hosts file by the query to http://www.yourdomain.com.
15. Set a Download Rate Limitation
You can set a download rate restriction of 100 KB/s for curl to avoid using your Internet.
$ curl --limit-rate 100K http://yourdomain.com/yourfile.tar.gz -OAlso Read: 14 Tar Command in Linux
Conclusion
You must know how to utilize the Linux curl command and all of its options after reviewing this article.
The fundamental application of Curl was demonstrated in this post. The most widely used HTTP methods, GET and POST, are covered in our instructions. We learned how to submit various sorts of data in a POST request, how to download a file, and how to modify a request’s header parameters. In this instance, we introduced Curl, a really important utility.
