Introduction
This came to my attention when I was trying out RabbitMQ, that powershell\s invoke-restmethod did not work as intended. Some REST calls worked, but some didn’t. It took me quite some time to figure out what was going on. Most information in this blog post is coming from one source, so it does not really add much to the internet. I need this to keep track, just in case the gitrepo or blog post below disappears. =)
TL;DR: Powershell strips away some characters from a URL/URI, even if you have URL encoded them.
Characters that are stripped away (perhaps even more are stripped away):
slash - /
trailing dot - .
So, what you think that you send to your RabbitMQ is not what RabbitMQ receives.
Quick background; in RabbitMQ, there are three levels of indirection:
- Vhost - defaults to “/” (which URL encodes to
%2f) - Exchange - defaults to amq.default
- Queue - you give it a name, and it works
Most people never have to care much about the vhost, as it is normally not required to set it to anything but the default. In most examples in the getting started documentation, you just see the “%2f” somewhere in some of the examples, and you don’t really care much about hit. This is where things start going bad if you have clients on Windows.
Proving my point
If you would like to publish a message to a queue, two of the necessary parameters can be more or less ignored; the Vhost and the Exchange. You usually need to select a Queue. In my example I use “MFT”, which stands for “managed file transfer”. I want to notify the file transfer application that something is ready to be picked up.
A typical REST call for this would be to POST to http://rabbitmq.host.local:15672/api/exchanges/%2f/amq.default/publish with the proper payload, say: {"properties":{},"routing_key":"MFT","payload":"INFRA-2036 ready","payload_encoding":"string"}.
You would expect your powershell command invoke-restmethod to do just that. But nooo.
What the server should receive:
10.20.30.40 - - [01/Dec/2019 17:34:21] "POST /api/exchanges/%2f/amq.default/publish HTTP/1.1" 401 -
What your server actually receives:
10.20.30.40 - - [01/Dec/2019 17:34:21] "POST /api/exchanges///amq.default/publish HTTP/1.1" 401 -
The difference is that instead of /%2f/ (which represents the Vhost “/"), invoke-restmethod parses the URL-encoded %2f and replaces it with /.
Test this, and you will see that powershell is doing the evil replacement:
PS C:\Users\myuser> $url="http://apa.bepa.com/asdf/%2f/."
PS C:\Users\myuser> $uri = new-object uri($url)
PS C:\Users\myuser> $uri.AbsoluteUri
http://apa.bepa.com/asdf///
Quick and dirty solution
Note: This code is copied from the git repo mentioned below. Most of my information is coming from the accompanying blog post from the same author.
#--- from https://github.com/mariuszwojcik/RabbitMQTools/blob/master/PreventUnEscapeDotsAndSlashesOnUri.ps1
#--- https://www.mariuszwojcik.com/how-to-prevent-invoke-restmethod-from-un-escaping-forward-slashes/
if (-not $UnEscapeDotsAndSlashes) { Set-Variable -Scope Script -name UnEscapeDotsAndSlashes -value 0x2000000 }
function GetUriParserFlags
{
$getSyntax = [System.UriParser].GetMethod("GetSyntax", 40)
$flags = [System.UriParser].GetField("m_Flags", 36)
$parser = $getSyntax.Invoke($null, "http")
return $flags.GetValue($parser)
}
function SetUriParserFlags([int]$newValue)
{
$getSyntax = [System.UriParser].GetMethod("GetSyntax", 40)
$flags = [System.UriParser].GetField("m_Flags", 36)
$parser = $getSyntax.Invoke($null, "http")
$flags.SetValue($parser, $newValue)
}
function PreventUnEscapeDotsAndSlashesOnUri
{
if (-not $uriUnEscapesDotsAndSlashes) { return }
Write-Verbose "Switching off UnEscapesDotsAndSlashes flag on UriParser."
$newValue = $defaultUriParserFlagsValue -bxor $UnEscapeDotsAndSlashes
SetUriParserFlags $newValue
}
function RestoreUriParserFlags
{
if (-not $uriUnEscapesDotsAndSlashes) { return }
Write-Verbose "Restoring UriParser flags - switching on UnEscapesDotsAndSlashes flag."
try
{
SetUriParserFlags $defaultUriParserFlagsValue
}
catch [System.Exception]
{
Write-Error "Failed to restore UriParser flags. This may cause your scripts to behave unexpectedly. You can find more at get-help about_UnEsapingDotsAndSlashes."
throw
}
}
if (-not $defaultUriParserFlagsValue) { Set-Variable -Scope Script -name defaultUriParserFlagsValue -value (GetUriParserFlags) }
if (-not $uriUnEscapesDotsAndSlashes) { Set-Variable -Scope Script -name uriUnEscapesDotsAndSlashes -value (($defaultUriParserFlagsValue -band $UnEscapeDotsAndSlashes) -eq $UnEscapeDotsAndSlashes) }
#=======================
# MAIN
#=======================
PreventUnEscapeDotsAndSlashesOnUri
$username = "ft"
$password = "ft"
$url = "http://rabbitmq.server.local:15672/api/exchanges/%2f/amq.default/publish"
$RoutingKey = "MFT"
$payload = "INFRA-2036 ready"
$Properties = @{}
$body = @{
routing_key = $RoutingKey
payload_encoding = "string"
payload = $Payload
properties = $Properties
} | ConvertTo-Json
#========== doit
$password = ConvertTo-SecureString $password -AsPlainText -Force
$Credentials = New-Object System.Management.Automation.PSCredential ($username, $password)
$result = Invoke-RestMethod $url -Method POST -Body $body -ContentType 'application/json' -Credential $Credentials
RestoreUriParserFlags
Even quicker solution
Instead of using invoke-restmethod, use curl.
choco install curl
curl -i -u ft:ft -XPOST -H "Content-Type: application/json" --data "{\"properties\":{},\"routing_key\":\"MFT\",\"payload\":\"INFRA-1909 ready\",\"payload_encoding\":\"string\"}" http://rabbitmq.server.local:15672/api/exchanges/%%2f/amq.default/publish