Unable to scrape data via request module - Fobidden
javascript,node.js,node-request
The server is also looking for an Accept header. So try adding something like 'Accept': 'text/html;q=0.9,*/*;q=0.8', to your headers object.
browserify watchify produces errors
node.js,gulp,browserify,node-request
It looks like it is an issue with elliptic, a dependency of browserify. See https://github.com/indutny/elliptic/issues/30 Hopefully this gets fixed up shortly. Edit: It may actually be an issue with browserify. I would try reverting to 8.1.2 and seeing if that helps....
Correspondence between curl command and node's request
node.js,curl,http-post,node-request
By default, cURL will send a Content-Type: application/x-www-form-urlencoded unless you use -F (which changes it to Content-Type: multipart/form-data) for your fields or explicitly override the header (e.g. -H 'Content-Type: application/json'). However, the data being sent by your cURL example seems to be JSON. So the server will get confused and...
Async web requests are making 'socket hangup' in node.js
You should look at option pool when send a request using request module. There are 3 options: Set it false Increase its maxSockets Increase http.globalAgent.maxSockets Examples: for (var i = 0; i < 10000; i++) { // make a request request({ pool: false, // other options }, function(err, res, body)...
NodeJS, Express and routing with external requests
You might find async.js to be useful in keeping your code clean from the "callback hell." Specifically the parallel or series methods. For example (not tested): async = require('async'); app.get('/profile', isLoggedIn, function(req, res) { async.parallel({ records: function(callback) { Object.find().exec(function (err, records) { callback(null, records); }); }, responseData1: function(callback) { request('firstUrl',...
Piping images versus sending callback body in node.js with request
javascript,node.js,http,request,node-request
The problem is that body is a (UTF-8) string by default. If you're expecting binary data, you should explicitly set encoding: null in your request() options. Doing so will make body a Buffer, with the binary data intact: request.get(imgUrl, { encoding: null }, function(err, resp, body) { if(err) { throw(err);...
Check app running on new port
javascript,node.js,express,node-http-proxy,node-request
There are few examples on various usage of proxy server. Here is a simple example of a basic proxy server: var http = require("http"); var httpProxy = require('http-proxy'); /** PROXY SERVER **/ var proxy = httpProxy.createServer({ target:'http://localhost:'+3000, changeOrigin: true }) // add custom header by the proxy server proxy.on('proxyReq', function(proxyReq,...
Node async.series trouble
node.js,node-async,node-request
You need to call the callback parameter that's passed into the functions of the async.series call when each function's work is complete. That's how async.series knows that it can proceed to the next function. And don't redefine readabilityData as a function parameter when you're trying to use it to share...
Streaming an uploaded file to an HTTP request
node.js,express,node-request,busboy,wistia
I am not to familiar with the busboy module, but there errors you are getting are from attempting to use un-implemented streams. Whenever you create a new readable or writable stream directly from the stream module you have to create the _read and _write methods respectively Stream Implementors (node.js api)....
How to make parameters Optional in node JS rest API
you need to structure the route like this: app.get('path/:required/:optional?*, ...) ...
Download an image using node-request, and fs Promisified, with no pipe in Node.js
node.js,promise,bluebird,fs,node-request
You have to tell request that the data is binary: requestAsync(uri, { encoding : null }) Documented here: encoding - Encoding to be used on setEncoding of response data. If null, the body is returned as a Buffer. Anything else (including the default value of undefined) will be passed as...
How do I encode an array in a form field in NodeJS' request?
php,arrays,node.js,form-data,node-request
In PHP arrays can be arrays and they can be associative arrays, which are essentially objects in JavaScript. Specifying with array() An array can be created using the array() language construct. It takes any number of comma-separated key => value pairs as arguments. array( key => value, key2 => value2,...
Fire a GET request and get the node Stream
Take a look this part of the docs. What you're describing is essentially this block request.get('http://google.com/img.png').pipe(request.put('some_url')); Note the docs When doing so, content-type and content-length are preserved in the PUT headers. Also note, request will always return a Stream if you don't specify a callback. If you do provide a...
pass JSON to HTTP POST Request
json,node.js,curl,express,node-request
I think the following should work: // fire request request({ url: url, method: "POST", json: true, headers: { "content-type": "application/json", }, body: JSON.stringify(requestData) }, ... Another solution according to https://www.npmjs.com/package/request#request-options-callback would be to use // fire request request({ url: url, method: "POST", json: requestData }, ... In this case, the...
Error: CERT_HAS_EXPIRED in Node.js request module (macu vs facebook)
node.js,ssl,ssl-certificate,node-request
Your intermediate certificate for DigiCert High Assurance EV Root CA seems to have expired, see f.e. https://www.sslshopper.com/ssl-checker.html#hostname=www.macu.com Likely your browser did not complain about it, because it had a newer, valid version of that intermediate cert installed already (and used that to prove the identity of the signing institution), whereas...
What is the equivalent of curl --upload-file in node-request
I threw up an example on Github that works. The gist is that you need to pipe the file into Request: fs.createReadStream(filePath).pipe(request.put(putURL,options,function(err, httpsResponse, body){ if ( err ) { console.log('err', err); } else { console.log(body); } })); Still not sure how to pass in a file stream as an option...