7 Linux cURL command Examples - How to test RESTful Web Services using in UNIX

The curl or cURL command of Linux is a very powerful and versatile command which allows you to send sophisticated HTTP request right from your Linux command line window. You can use the cURL command to test your RESTful Web Services by sending GET and POST request, doing authentication, saving a cookie in the file etc. The curl command is, in fact, the go-to tool for many Java and Spring developers working in web application and consuming data from the secured RESTful Web Services, but you can also use it test your simple REST Web APIs without security. I have used it many times to check if our Web service is up or not, or is there any error while accessing those services.

You can also use this to write a shell script for automatically doing health-check for your environment. All you need to do is to schedule the shell script using crontab command and it will automatically send a request and report errors or success at a fixed time of day.

Here are some of the useful command line options of cURL command, which is extremely important for connecting and testing your RESTful Web Service:

[--digest] it's a digest authentication
[-u{username}:{password}] attaching username and password
[-X PUT] method="put"
[-H 'Expect: '] header = 'Expect: '
[-H 'Content-type: application/xml'] additional header

The syntax of curl command is as follows:

$ curl --digest \
-u{username}:{password} \
-v \
-X PUT \
-H 'Expect: ' \
-H 'Content-type: application/xml' \
-d @- \
http://localhost:8080/SpringRestDemo/api/book/9783827321312 \
< data.xml 

Here, d is for data. The "-d@ -" option will instruct curl command to send a POST request with the data it reads from standard input. The '<' operator tells the shell to feed a file to stdin. You can make it simpler by doing -d @data.xml and not use standard input at all.

Btw, if you are new to Linux, I suggest you go through Learn Linux in 5 Days and Level Up Your Career course on Udemy to get a head-start and learn some fundamentals + essential Linux commands you will going to use every day. It's an awesome course and very affordable too. I bought in just $10 on Udemy's flash sale which happens every now and then.





Testing RESTful Web Services from Linux command line 

Here is my list of some of the most useful examples of curl command, which I use in my day to day life to test RESTful web services from the command line. You can even use these to write scripts and run it from crontab to automatically test the availability of your RESTful API, very useful if you are supporting a production REST application.


1) How to test authentication against REST API

Suppose the URL for login in your REST web service is http://localhost:8080/SpringRestDemo/j_spring_security_check then you can use the following curl command for performing login:

$ curl -i -X POST -d j_username=user -d j_password=password
http://localhost:8080/SpringRestDemo/j_spring_security_check

This request will also return the Cookie which will then be used by any subsequent request against the same REST Web Service.

Btw, I have used Spring security to protect our RESTful web service here, if you want to learn more about that, you can check Learn Spring Security: The Certification Class by Eugen Paraschiv of Baeldung.   It's one of the better course to learn Spring Security, which provides a guided and code-focused tour of Spring Security. 

 7 Linux cURL command Examples - How to test RESTful Web Services using in UNIX


2) Saving the cookie in a file using the curl command

While authenticating against RESTful Web Service, If you want, you can also save the cookie into a specific file using the curl command as shown below:

$ curl -i -X POST -d j_username=user -d j_password=password -c /tmp/cookies.txt
http://localhost:8080/SpringRestDemo/j_spring_security_check

This request will save the Cookie into /tmp/cookies.txt file.

You can also save the cookie into your home directory if you like, it will be same there as /tmp is generally accessible to everybody and also it's frequently cleaned by Linux.



3) Attaching header and Cookie into HTTP request using Curl

You can also attach HTTP headers using curl command by using option --header for authentication and authorization purposes. You can even attach cookie into your HTTP request using -b command option as shown below:

$ curl -i --header "Accept:application/json" -X GET -b /tmp/cookies.txt
http://localhost:8080/SpringRestDemo/j_spring_security_check


This request will attach "Accept" header and earlier saved cookie from /tmp/cookies.txt file into your HTTP request. The response will look like below:

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 06 Jun 2017 22:21:23 IST
[{"ISBN":9783827321312,"title":"Effective Java"}]

If you are not very familiar with HTTP but currently working on a project which has RESTful web service, I strongly suggest you to first go through HTTP Fundamentals course at Pluralsight, it's a free course and give you enough knowledge about HTTP to survive in your day job while working with HTTP and REST.

testing RESTful web service from command line in UNIX





4) Accessing RESTful Web Service 

If your RESTful Web Service doesn't implement security then you can access a resource using curl command as shown below:

$ curl -i http://localhost:8080/SpringRestDemo/api/book/9783827321312

This will return the JSON representation of book with ISBN 9783827321312, but if your REST API is secured e.g. by using http basic auth, then you will receive a 403 unauthrorized response as shown below:

HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=25A833C16B6530A007CFA3AECEE2216B; Path=/SpringRestDemo/; HttpOnly
WWW-Authenticate: Basic realm="Secure REST API"
Content-Type: text/html;charset=utf-8
Content-Length: 1028
Date: Tue, 06 Jun 2017 22:21:23 IST

If you test the same REST API using a browser you will be prompted to enter username and password by the browser because it will use HTTP basic authentication, but curl won't do that. You need to specially provide it username and password as shown in the next example. See REST Fundamentals By Howard Dierking to learn more about HTTP basic and digest authentication in REST, it's a free course and you can finish it a couple of hours.





5) Accessing Secure RESTful Web Service using username and Password


You can use the --user command line option of curl to send username and password along with HTTP request to access a secure REST web API as shown below:

$ curl -i --user username:password http://localhost:8080/SpringRestDemo/api/book/9783827321312

Now, you will receive an HTTP response with success code 200, along with a cookie. This is useful when your application is using username and password stored in a database or flat file for login.

It can also be used in conjunction with LDAP based authentication, as long as you just need to provide username and password.



6) Enabling digest authentication using the curl command

If your REST API is secured using digest authentication then you can use --digest flag to enable HTTP digest authentication in curl command as well.

$ curl --digest 
       --user username:password
        -i
 http://localhost:8080/SpringRestDemo/api/book/9783827

Btw, if you are curious about how to secure your API using digest authentication, well, you can use Spring security. It supports both HTTP basic and digest authentication. You can see Learn Spring Security for implementation details.




7) Setting more than one header in the curl command

If you want you can set more than one HTTP header in your HTTP request by using -H command line option twice using curl in UNIX, as shown below:

$ curl -H "Accept: application/json"
       -H 'If-None-Match: "12334dfsfsdffe004fsdfds36a6"'
       -i http://localhost:8080/SpringRestDemo



That's all about how to use curl command to test RESTful Web Services. You can run these curl command from Linux command line window right from the server or by using Putty or Cygwin from your development machine. If you don't want to use the Linux command line, you can also use tools like Postman and some Chrome extension to test your REST API.

Further Learning
Linux Command Line Basics
Linux Command Line Interface (CLI) Fundamentals
Learn Linux in 5 Days and Level Up Your Career
REST API Automation testing from scratch-(REST Assured java)


Other REST and Linux command tutorials you may like
  • How does nslookup command work in UNIX? (answer)
  • 10 examples of lsof command in Linux? (examples)
  • 5 Books to learn RESTful Web Services  (list)
  • When to use PUT or POST in REST? (answer)
  • 10 Frequently asked RESTful Web Service Interview Questions (list)
  • How to send mail with attachments from Linux? (mailx)
  • 10 example of Networking commands in Unix (nslookup)
  • 5 Example of kill commands in Unix and Linux (example)
  • How to find all files matching specific text in Linux (grep)
  • How to use the netstat command to find which process is listening on a port? (example)
  • Linux find + du + grep example (example)
  • 10 Examples of chmod command in Linux (chmod)
  • A Practical Guide to Linux Commands, Editors, and Shell Programming (guide)

Thanks for reading this article, if you like this article then please share with your friends and colleagues. If you have any suggestion or feedback then please drop a comment.

P. S. - If you are looking for some free online courses to start your Linux journey, you should check out my list of Free Linux Courses for Programmers, Web Developers, IT Professionals, and System Administrators.

1 comment: