HTTP limitations? (DEF-3410) (SOLVED)

#1

I am working on integrating Firebase into my game, which means creating a library for the REST API. I’ll gladly share it in the future when its done.

I’m having an issue with a request that I can get to work in Postman, but the identical request in Defold doesn’t seem to work.

I am making a PUT request to a URL, the payload seems to pass Firebase’s checks (good), but when it comes to authentication… not so much. To authenticate with the Firebase REST API you must append ?auth=KEY to your requests, where KEY is an authentication token generated via another REST authentication method.

The key is fine, yet Firebase complains that it cannot parse the key, yet when I make the identical PUT in Postman - it works just fine.

My initial thought was that Defold is doing some magic to the URL which is corrupting the key, because if I pass an invalid or malformed key to the REST path (in Postman) it gives me the same error.

https://FIREBASE/user_saves/X4Rj7Jht8EfLVJWgOLOW6Lg5hbs1/save_file.json?auth=A_VERY_VERY_VERY_LONG_KEY

Hopefully (edit: somebody!) has some clues on this one :slight_smile:

#2

I can happily share some keys/urls if anybody would like to verify I am not going out of my mind, only thing though is that the tokens expire after an hour. I am in London time zone and will probably be signing off for the day now :slight_smile:

#3

I can assist. I’m thinking that it has something to do with URL length. We have increased max length once in the past when I did some work on the AWS SDK where Gamelift URLs became really long.

#6

OK I’ll get some keys generated and URL to use, do you want to take it into DMs? Let me know.

Edit: Really messed up that reply…! :smiley:

#7

In case we miss each other in real time @britzl - is there anything I can do on my end if the problem is related to URL length? I’m trying to see if I can send the data I need in headers or some other workaround - but this is very much dependant on the available API Firebase provide.

#8

What’s the length of a complete URL with all parameters added?

#9

@britzl Over 1000 characters, anywhere between 1000-1100.

#10

Hmm, ok, from what I can tell the max URI length we support is 2048, so that should be enough then.

Could it be some encoding problem? Do you need to URL encode any of the arguments?

#11

The URL looks like this:

https://triggered-d1327.firebaseio.com/test/anything/some_value.json?auth=eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmlyZWJhc2UiOnsiaWRlbnRpdGllcyI6eyJmYWNlYm9vay5jb20iOlsiMTAxNTYzNjg4NDU2NzU0MzUiXX0sInNpZ25faW5fcHJvdmlkZXIiOiJmYWNlYm9vay5jb20ifX0.LlCJP8yYHYV7P6cZRGs1iIipRNrXZIJ0qM2Lpxf7sC6FLnQsT1kSlBw7UqT32aoNe0Q4ckjet8YMAT1ndsprGM8KaZq7vYHvx2tQRo5ej81LWc1U4fHHWbFChPOMbD-QPTDrQUilKyiADX21Vhpynvq0LhS-zizCDKfC2cWwCB3rzgCivojek4zf4sunlFXC7zY9OFrivH4f3uFdB97uUy2i-lnAb2sY4_vRTvXg-pdymqUP6mL6fMlaQAzI5exZ-Dqc62jZfT846cF6ks1dnEy4s9m6LsLiz3qtHrP5RafhBd-MkA6-9TXR01nzVu_bjls6v-FUvayxAtqBkxHoxA

If you make this as a GET request, you’ll get a response

{
  "error" : "Auth token is expired"
}

If you try the same in Defold (I hope) you will get the error I get about not being able to parse the auth key.

#12

For clarity:

The code:

local url = "https://triggered-d1327.firebaseio.com/test/anything/some_value.json?auth=eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmlyZWJhc2UiOnsiaWRlbnRpdGllcyI6eyJmYWNlYm9vay5jb20iOlsiMTAxNTYzNjg4NDU2NzU0MzUiXX0sInNpZ25faW5fcHJvdmlkZXIiOiJmYWNlYm9vay5jb20ifX0.LlCJP8yYHYV7P6cZRGs1iIipRNrXZIJ0qM2Lpxf7sC6FLnQsT1kSlBw7UqT32aoNe0Q4ckjet8YMAT1ndsprGM8KaZq7vYHvx2tQRo5ej81LWc1U4fHHWbFChPOMbD-QPTDrQUilKyiADX21Vhpynvq0LhS-zizCDKfC2cWwCB3rzgCivojek4zf4sunlFXC7zY9OFrivH4f3uFdB97uUy2i-lnAb2sY4_vRTvXg-pdymqUP6mL6fMlaQAzI5exZ-Dqc62jZfT846cF6ks1dnEy4s9m6LsLiz3qtHrP5RafhBd-MkA6-9TXR01nzVu_bjls6v-FUvayxAtqBkxHoxA"
	http.request(url, "GET", function(_,_,response)
		pprint(response)
	end)

Gives the following output:

{
  status = 401,
  response = {
  "error" : "Could not parse auth token."
}
,
  headers = {
    server = nginx,
    strict-transport-security = max-age=31556926; includeSubDomains; preload,
    content-type = application/json; charset=utf-8,
    connection = keep-alive,
    content-length = 46,
    cache-control = no-cache,
    date = Fri, 17 Aug 2018 16:55:17 GMT,
    access-control-allow-origin = *,
  }
}

But paste that into a browser, or into Postman, and you’ll get:

{
  "error" : "Auth token is expired"
}
#13

Interesting. If I build and launch in HTML5 it works and on my Mac it doesn’t. We do use a different implementation of http.request() on HTML5 though.

Have you tried to capture the data that is being sent to Firebase?

#14

No, I haven’t. Interesting you say that though because I am using a Mac, as well as streaming the build to an iPhone (same result).

Been a while since I’ve used a data sniffer, I wonder if Charles is still up to the task. I’ll take a look.

#15

Charles will not work. You need to use a transparent proxy working on a low level.

#16

Yes I am noticing that :slight_smile:

#18

OK, I used postman echo.

For the request:

local url = "https://postman-echo.com/get?auth=eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmlyZWJhc2UiOnsiaWRlbnRpdGllcyI6eyJmYWNlYm9vay5jb20iOlsiMTAxNTYzNjg4NDU2NzU0MzUiXX0sInNpZ25faW5fcHJvdmlkZXIiOiJmYWNlYm9vay5jb20ifX0.LlCJP8yYHYV7P6cZRGs1iIipRNrXZIJ0qM2Lpxf7sC6FLnQsT1kSlBw7UqT32aoNe0Q4ckjet8YMAT1ndsprGM8KaZq7vYHvx2tQRo5ej81LWc1U4fHHWbFChPOMbD-QPTDrQUilKyiADX21Vhpynvq0LhS-zizCDKfC2cWwCB3rzgCivojek4zf4sunlFXC7zY9OFrivH4f3uFdB97uUy2i-lnAb2sY4_vRTvXg-pdymqUP6mL6fMlaQAzI5exZ-Dqc62jZfT846cF6ks1dnEy4s9m6LsLiz3qtHrP5RafhBd-MkA6-9TXR01nzVu_bjls6v-FUvayxAtqBkxHoxA"

The response was:

{
  status = 200,
  response = {"args":{"auth":"eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmlyZW"},"headers":{"host":"postman-echo.com","x-forwarded-port":"443","x-forwarded-proto":"https"},"url":"https://postman-echo.com/get?auth=eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmlyZW"},
  headers = {
    server = nginx,
    content-length = 1156,
    content-type = application/json; charset=utf-8,
    connection = keep-alive,
    date = Fri, 17 Aug 2018 18:41:36 GMT,
    vary = Accept-Encoding,
    set-cookie = sails.sid=s%3AbRAlWCL1aNWCW5UO35xTDWuxlvEUK6Ty.yB%2FSUkicRkENPzWC5spZftydEZW5Teu1hiNArNkLrkU; Path=/; HttpOnly,
    etag = W/"484-NYDoATF7ogx14SCEpR3XSNw4c7A",
  }
}

It seems that postman echo is not receiving anything after:

https://postman-echo.com/get?auth=eyJhbGciOiJSUzI1NiIsImtpZCI6IjY3NmY5MmU1MGQ5ZmUxNzdiM2I5NTJjNGM4ZWU2YjY1ZDk3ZWIwZmMifQ.eyJpc3MiOiJodHRwczovL3NlY3VyZXRva2VuLmdvb2dsZS5jb20vdHJpZ2dlcmVkLWQxMzI3IiwibmFtZSI6IkpvbmF0aGFuIEhhcnJpcyIsInBpY3R1cmUiOiJodHRwczovL2dyYXBoLmZhY2Vib29rLmNvbS8xMDE1NjM2ODg0NTY3NTQzNS9waWN0dXJlIiwiYXVkIjoidHJpZ2dlcmVkLWQxMzI3IiwiYXV0aF90aW1lIjoxNTM0NTE3MzEyLCJ1c2VyX2lkIjoiWDRSajdKaHQ4RWZMVkpXZ09MT1c2TGc1aGJzMSIsInN1YiI6Ilg0Umo3Smh0OEVmTFZKV2dPTE9XNkxnNWhiczEiLCJpYXQiOjE1MzQ1MTczMTIsImV4cCI6MTUzNDUyMDkxMiwiZmly

which leaves out a good portion of the URL:

ZWJhc2UiOnsiaWRlbnRpdGllcyI6eyJmYWNlYm9vay5jb20iOlsiMTAxNTYzNjg4NDU2NzU0MzUiXX0sInNpZ25faW5fcHJvdmlkZXIiOiJmYWNlYm9vay5jb20ifX0.LlCJP8yYHYV7P6cZRGs1iIipRNrXZIJ0qM2Lpxf7sC6FLnQsT1kSlBw7UqT32aoNe0Q4ckjet8YMAT1ndsprGM8KaZq7vYHvx2tQRo5ej81LWc1U4fHHWbFChPOMbD-QPTDrQUilKyiADX21Vhpynvq0LhS-zizCDKfC2cWwCB3rzgCivojek4zf4sunlFXC7zY9OFrivH4f3uFdB97uUy2i-lnAb2sY4_vRTvXg-pdymqUP6mL6fMlaQAzI5exZ-Dqc62jZfT846cF6ks1dnEy4s9m6LsLiz3qtHrP5RafhBd-MkA6-9TXR01nzVu_bjls6v-FUvayxAtqBkxHoxA
#19

Yes, I did a similar test and found that we limit the path to 512 characters. This obviously needs to be fixed. I created a ticket to track this: DEF-3410. It should be an easy fix but I can’t make any promises as to when we will release it.

Does Firebase support an auth token header perhaps?

#20

Yes, it’s basically this:

char m_Path[512];

Should only be a matter of increasing to something a bit larger, say 2048. 2048 seems like a reasonable value based on this: https://stackoverflow.com/a/417184/1266551

#21

I’ll take a look into workarounds - but unfortunately I don’t think they do.

At least I know what’s causing the problem and if you have a fix in the works I can sleep easy!

Thanks

#22

This page seems to indicate support for an auth header, but I couldn’t get it to work:

https://firebase.google.com/docs/database/rest/auth#authenticate_with_an_access_token

Am I looking at the wrong docs?

#23

That’s the right page, but there are two ways to authenticate.

You can use a header to authenticate with Google OAuth2 access tokens, typically used on a server - which gives you full read and write access to the database.

If you want to authenticate with Firebase ID tokens, which are used for authenticating users - you must append the auth parameter to the request.