Answer by Oleg Valter is with Ukraine for HTTP request with XML payload using UrlFetchApp

Problem

You are passing an invalid second argument to the fetch() method call.

Solution

There is a closed set of acceptable parameters that you can pass in the configuration object. Please, make sure that names of the parameter are exactly the same as in the reference (see below). In your case, content-type should be renamed to contentType (currently, your request sends your XML document as application/x-www-form-urlencoded).

Consider using a muteHttpExceptions set to true to ease debugging process in the future.

Update

After some more thorough investigation, the error you are experiencing is highly likely to be related (though previous issue still stands) to passing a parsed XML document (e.g. created via XmlService) to the payload if that is what you are doing (the error is reproduced consistently under this condition). Note that payload can accept either a String or byte[] or Blob instance or Object (varies on contentType parameter).

Modification

As per Tanaike's comment and suggestion, if you are indeed trying to pass a Document instance to the payload, the issue will be amended by preparing the XML (note that these operations don't mutate xml value), for example:

1. XmlService.getPrettyFormat().format(XmlService.parse(xml));
2. XmlService.getRawFormat().format(xml) (if XML is already parsed);

Useful links

1. fetch() method reference;
2. XmlService class reference;
3. getRawFormat() method reference (just in case);