POST large JSON Data without cURL call in Node.js

Tudip

22 March 2018

Basic cURL information:

Curl is used to transfer data to a server and also for to get data from the server, using one of the supported protocols (HTTP, HTTPS, FTP). With cURL we can do things likes proxy support, user authentication, file uploading, HTTP posting, SSL connections, and many other things. cURL can be used in many different ways. But there are some limitation in cURL call in Node.js and they are as below:

Limitation of cURL POST call in Node.js:

When we use ‘curlrequest’ npm module in Node, there are some limitations (like you can’t pass data more than 100KB through cURL in Node.js) so to overcome these limitations, we can use ‘request’ npm module to call third-party APIs. And When we try to send a smaller file (with a smaller body) using cURL call it works. When we send the request using Postman it works well even with the large file. But the problem with cURL is when we try to send large JSON data through cURL call in Node.js then it gives following error:

internal/child_process.js:319
 throw errnoException(err, 'spawn');
 ^
Error: spawn E2BIG
 at exports._errnoException (util.js:1050:11)
 at ChildProcess.spawn (internal/child_process.js:319:11)
 at Object.exports.spawn (child_process.js:378:9)
 at module.exports (/Users/Leon/Desktop/demo-repository/node_modules/curlrequest/spawn.js:19:30)
 at Object.exports.request (/Users/Leon/Desktop/demo-repository/node_modules/curlrequest/index.js:248:17)
 at Object. (/Users/Leon/Desktop/demo-repository/test.js:40:10)
 at Module._compile (module.js:571:32)
 at Object.Module._extensions..js (module.js:580:10)
 at Module.load (module.js:488:32)
 at tryModuleLoad (module.js:447:12)

Use of ‘request’ NPM module in Node.js:

To solve above issue, we can use ‘request’ npm module to send large JSON data through Node.js without using cURL. ‘request’ library is much more user-friendly than the default HTTP module and it has been used in Node.js community for several years. You can install it as a dependency from npm in your Node.js project. Run the following in your terminal from the directory you want your code to live in:

npm install request

How ‘request’ module works in Node.js:

Connecting to external APIs is easy in Node. You just require the core basic HTTP module and start calling external APIs. There are many ways to call external APIs. On NPM you can find multiple modules that can help you to call third-party APIs. For example, “request” and “superagent” modules are used to call external APIs.

Use of Request Module in Node.js:

1. 1. Sending a GET request with sample code:

      const options = {
       method: 'GET',
       url: '----API url------',
       headers: {
      Authorization: "Basic " + new Buffer("---API credentials---").toString("base64")
       },
      }
      request(options)
       .then(function (response) {
       // Handle success response data
       })
       .catch(function (err) {
       // Handle err response
       })
      

   2. Sending a POST request with sample code:

      const options = {
       method: 'POST',
       url: '----API url------',
       headers: {
      Authorization: "Basic " + new Buffer("---API credentials---").toString("base64")
       },
       body: {
       // send body data here
       }
       json: true // JSON stringifies the body automatically
      }
      request(options)
       .then(function (response) {
       // Handle success response data
       })
       .catch(function (err) {
       // Handle err response
       })
      

   3. Also, you can add query string parameters, you just have to add the qs property to the options object:

      const options = {
       method: 'GET',
       url: '----API url------',
       qs: {
       limit: 10,
       skip: 20,
       sort: 'asc'
       }
      }
      

Note: Error handling Error handling is an essential part while making requests to external APIs, as we can never be sure what will happen to them. Apart from our client errors the server may respond with a different error or can different data in different format. Keep these things in mind when you try to handle the external APIs response. Also, it is good to use a catch block to avoid external APIs response.

Reference:

• https://www.npmjs.com/package/curlrequest
• https://www.npmjs.com/package/request