python - Newlines removed in POST request body? (Google App Engine) -

i building rest api on google app engine (not using endpoints) allow users upload csv or tab-delimited file , search potential duplicates. since it's api, cannot use <form>s or blobstore's upload_url. cannot rely on having single web client call api. instead, ideally, users send file in body of request.

my problem is, when try read content of tab-delimited file, find newline characters have been removed, there no way of splitting content rows.

if check content of file directly on python interpreter, see tabs , newlines there (output truncated in example)

>>> open('./data/occ_sample.txt') o: ...     o.read() ...  'id\ttype\tmodified\tlanguage\trights\n123456\tphysicalobject\t2015-11-11 11:50:59.0\ten\thttp://creativecommons.org/licenses/by-nc/3.0\n...' 

the requesthandler logs content of request body:

import logging class reportapi(webapp2.requesthandler):     def post(self):         logging.info(self.request.body)         ... 

so when call api running in dev_appserver via curl

curl -x post -d @data/occ_sample.txt http://localhost:8080/api/v0/report 

this shows in logs:

id  type    modified    language    rights123456    physicalobject  2015-11-11 11:50:59.0   en  http://creativecommons.org/licenses/by-nc/3.0 

as can see, there nothing between last value of headers , first record (rights , 123456 respectively) , same happens last value of each record , first 1 of next.

am missing obvious here? have tried loading data self.request.body, self.request.body_file , self.request.post, , none seem work. tried applying content-type values text/csv, text/plain, application/csv in request headers, no success. should add different content-type?

you using wrong curl command-line option send file data, , this option stripping newlines.

the -d option parses out data , sends application/x-www-form-urlencoded request, , strips newlines. curl manpage:

-d, --data <data>

[...]

if start data letter @, rest should file name read data from, or - if want curl read data stdin. multiple files can specified. posting data file named 'foobar' done --data @foobar. when --data told read file that, carriage returns , newlines stripped out.

bold emphasis mine.

use --data-binary option instead:

--data-binary <data>

(http) posts data specified no processing whatsoever.

if start data letter @, rest should filename. data posted in similar manner --data-ascii does, except newlines , carriage returns preserved , conversions never done.

you may want include content-type header in case; of course depends on handler if care header.