Json encode / decode question

[Alex de Bruin 16]

I want to use a construction in my hometable like for example

 

local HT = { light1 = { fib_id = 211, hue_id = 7  },
                     light2 = { fib_id = 227, hue_id = 12 },
                   }
HT.ftoh  = { [ HT.light1.fib_id ] = HT.light1.hue_id,
                    [ HT.light2.fib_id ] = HT.light2.hue_id,
                   }

 

When i test fibaro:debug( HT.ftoh[ 227 ] ) it gives the expected value 12.

 

I want to use this in a couple of VD's with generic code for hue lights, but when i test local HT2 = fibaro.decode( fibaro.encode( HT ) ) then
fibaro:debug( HT2.ftoh[ 227 ] ) gives un expected NIL result.

 

fibaro:debug( HT2.ftoh[ "227" ] ) does give the correct result 12 so the original key index 227 is now a string.

 

Is this normal behaviour of the json.encode / json.decode functions or is this a bug ?

Edited March 8, 2020 by Alex de Bruin
Changed : to .

[Bodyart 226]

the correct syntax is:

 

jsonTable = json.decode(jsonString)
and

jsonString = json.encode(jsonTable )

 

Enjoy coding

[Alex de Bruin 16]

24 minutes ago, Bodyart said:

the correct syntax is:

 

jsonTable = json.decode(jsonString)
and

jsonString = json.encode(jsonTable )

 

 

My problem is that when i say

 

jsonStringX = json.encode( jsonTableX)

and then

jsonTableY = json.decode(jsonStringX)

 

then jsonTableY is not the same as the original jsonTableX

[Bodyart 226]

when i try this code:

Please login or register to see this code.

then i get:

Please login or register to see this image.

/monthly_2020_03/afbeelding.png.159614b253ab034b285a8358aab3f783.png" />

 

so the HT2 is exact same as HT, but you didn't define HT2.ftoh, only HT.ftoh......

Edited March 8, 2020 by Bodyart

[Mateo 2]

Hi, I do have an issue than may come from the same situation.

 

I've created in a scene a naming table for all my devices ID, rooms and so on and json.encode it into a global variable.

From the scene, i can then json.decode the variable and check that i can access my data. fine.

 

From a QA, i can't json.decode this global variable : "/5.3/json/util.lua:55: bad argument #1 to 'for iterator' (table expected, got number)"

trying numerous other expression format, i sometime get error return that let me think that data are not classified anymore, preventing from using them.

 

a little bit lost also...

 

any code angel ?

 

[Alex de Bruin 16]

1 hour ago, Bodyart said:

so the HT2 is exact same as HT, but you didn't define HT2.ftoh, only HT.ftoh......

 

I did define the HT.ftoh before encoding so it's part of the original table. After decoding it's part of the new HT2 table

 

Please login or register to see this image.

/monthly_2020_03/Knipsel.PNG.f20e1bce452a8c1fe679ec17e192a382.PNG" />

 

As you can see the original { [211]=7,[227]=12} is encoded as {"211":7,"227":12}

Edited March 8, 2020 by Alex de Bruin

[cag014 592]

3 hours ago, Alex de Bruin said:

I want to use a construction in my hometable like for example

 

local HT = { light1 = { fib_id = 211, hue_id = 7  },
                     light2 = { fib_id = 227, hue_id = 12 },
                   }
HT.ftoh  = { [ HT.light1.fib_id ] = HT.light1.hue_id,
                    [ HT.light2.fib_id ] = HT.light2.hue_id,
                   }

 

When i test fibaro:debug( HT.ftoh[ 227 ] ) it gives the expected value 12.

 

I want to use this in a couple of VD's with generic code for hue lights, but when i test local HT2 = fibaro.decode( fibaro.encode( HT ) ) then
fibaro:debug( HT2.ftoh[ 227 ] ) gives un expected NIL result.

 

fibaro:debug( HT2.ftoh[ "227" ] ) does give the correct result 12 so the original key index 227 is now a string.

 

Is this normal behaviour of the json.encode / json.decode functions or is this a bug ?

This is totally normal.

In both cases this is a string.

 Your definition using square brackets appends the value to string and json.encode function converts this to actual string.

HT.ftoh  = { [ HT.light1.fib_id ] = HT.light1.hue_id,
                    [ HT.light2.fib_id ] = HT.light2.hue_id,
                   }

You can print and see they are the same in json.encode

print(json.encode(HT))
  HT2=json.decode(json.encode(HT))
print(json.encode(HT2))

Edited March 8, 2020 by cag014

[Alex de Bruin 16]

36 minutes ago, cag014 said:

This is totally normal.

In both cases this is a string.

 

But why then HT.ftoh[ 227 ] = 12 before encoding and HT2.ftoh[ 227 ] = nil after encoding and decoding.

Edited March 8, 2020 by Alex de Bruin

[cag014 592]

1 minute ago, Alex de Bruin said:

But why then HT.ftoh[ 227 ] = 12 before encoding and HT2.ftoh[ 227 ] = nil after encoding and decoding.

json.encode converts the value to string, so json.decode gets back string value

[jgab 1,599]

9 minutes ago, cag014 said:

json.encode converts the value to string, so json.decode gets back string value

 

Which is kind of faulty.

It's a sparse array so a correct encoder should give an error like "invalid table: mixed or invalid key types"

Which the json encoder in scenes on the HC3 does, but not the json encoder in QA's on the HC3 (and not the HC2 scene version either)

If you don't get back what you encode when you decode it's not an encoder/decoder... it's something else.

It's possible to fool the json encoder on the HC2 too if you do a long sequential array 1,2,3,4,5...80 and then suddenly make huge holes...

[Alex de Bruin 16]

17 minutes ago, jgab said:

If you don't get back what you encode when you decode it's not an encoder/decoder... it's something else.

 

Thank you jgab, i agree.

It may be normal behaviour, but it should not be.

 

I use this table construction in my hometable and it's correct table usage according to the lua manual.

 

My purpose was to use the table in VD's where i use the selfID as an index to the corresponding hue id and was wondering why it did not work.

[jgab 1,599]

9 minutes ago, Alex de Bruin said:

 

Thank you jgab, i agree.

It may be normal behaviour, but it should not be.

 

I use this table construction in my hometable and it's correct table usage according to the lua manual.

 

My purpose was to use the table in VD's where i use the selfID as an index to the corresponding hue id and was wondering why it did not work.

 

To be more specific:

In json it is a difference between an array [...] and a key-value table {key:value}

Lua is a bit more .... "flexible"

 

Try this on the HC2.

Please login or register to see this code.

and you get an json array as expected '["A1","A2",...]'

If you instead do

Please login or register to see this code.

suddenly the encoder decides that it's a key-value table and you get '{"1"="A1","2"="A2",...."100"="HUPP"}'

Nice try but this does not make sense and creates strange bugs...

A strict (professional) json encoder tells the user that this can't be encoded  - actually I believe that some of the better ones can be told to be a bit relax and accept holes in arrays - but don't know what they do then... but changing the data types of the elements in the struct is a no-no.

 

[Alex de Bruin 16]

OK, that"s clear. I then have to use tostring(fibaro:getSelfId()) as the index in the decoded table or find a more elegant solution.

 

Thanks to all contributors to the subject.

[petergebruers 1,250]

8 minutes ago, Alex de Bruin said:

OK, that"s clear. I then have to use tostring(fibaro:getSelfId()) as the index in the decoded table or find a more elegant solution.

yes indeed and maybe define that 211 as a string because that's what it will be after a decode

 

I am a bit late to the party but I could not resist writing a little program to make this issue more visible...

 

Please login or register to see this code.

Result

 

Please login or register to see this code.

It is a game of "Can you spot the difference?"

 

Nice one!

[Mateo 2]

hello

 

a quick question regarding this json.encode thing

Trying to encode before storing in global variable, it can take up to 30 seconds before it's done.

i'm suspecting a sudden change of mind as evoked per @jgab

 

i've high level checked that i was not mising in my table numeric and string keys, but it remains anormally long even if what i store is a little bit more massive that usually (all hue details for 2 bidges, with groups, scenes and lights details)

 

Is it normal that json.decode become slow when some significant data? or is the issue in my table structure that is slowing the whole thing?

what are your reaction and advise?

 

thanks

 

[jgab 1,599]

33 minutes ago, Mateo said:

hello

 

a quick question regarding this json.encode thing

Trying to encode before storing in global variable, it can take up to 30 seconds before it's done.

i'm suspecting a sudden change of mind as evoked per @jgab

 

i've high level checked that i was not mising in my table numeric and string keys, but it remains anormally long even if what i store is a little bit more massive that usually (all hue details for 2 bidges, with groups, scenes and lights details)

 

Is it normal that json.decode become slow when some significant data? or is the issue in my table structure that is slowing the whole thing?

what are your reaction and advise?

 

thanks

 

No it doesn't. It's fast. Show your code.  

[Mateo 2]

 

 

22 minutes ago, jgab said:

No it doesn't. It's fast. Show your code.  

 

It's a little bit long, may I ?

[Mateo 2]

4 hours ago, Mateo said:

 

 

 

It's a little bit long, may I ?

 

i try  fighting with it for so long hours now ... i'm missing something ...

 

The purpose of the code is:

* grabbing all hue details from 2 bridges to store it in fibaro to feed other scene and QA 

* hue authentication and other association between hue Group and Fibaro rooms comes from a gobal variable where i store all details devices

* i access hue bridge 3 times for groups, scenes and lights (coming)

* i complete each hue dataset with new value to store in it associated bridge and the name of the fibaro room

* I build a new table structured with the fibaro room name, 3 sub sections for groups, scenes and lights and in there, the details from hue

* at the exception of the scenes that i encapsulate into a list from 1 to XXX to align it to the different period of the day that on which all my scenario are based on.

 

I'm suspecting mixing some numeric key with string one but can't find where, testing with "type" every key i can...

desesperate

 

thanks in advance @jgab 

 

 

 

Please login or register to see this code.

 

Edited March 28, 2021 by Mateo

[jgab 1,599]

I suspect that the code crashes and that's why you see such delays. I can't run your code as I'm lacking the configuration (indexTable) but try the version here that wraps the success handlers in pcall and logs ev. errors.

Please login or register to see this spoiler.

 

...also, avoid nesting global functions inside the success handler. The code becomes difficult to read (many levels) and they share index variables like 'I' and 'k' that is a recipe for future bugs... (http:request is also asynchronous - sharing globals in asynchronous calls is dangerous)

Many <font tags are not terminated which kind of works as the console is forgiving, however,  my emulator expects well formed html and had difficulties...  Also, putting html codes in the debug tag is a bit "novel".

 

Edited March 29, 2021 by jgab

[petergebruers 1,250]

I wonder if the http calls time out for some reason. We cannot know right now because there is no "error" part in the request of the call. Add something like this:

 

Please login or register to see this code.

 

You need more error handling than that but it is a start.

[jgab 1,599]

6 minutes ago, petergebruers said:

I wonder if the http calls time out for some reason. We cannot know right now because there is no "error" part in the request of the call. Add something like this:

 

Please login or register to see this code.

 

You need more error handling than that but it is a start.

In my version of his code that I attached I have added the error handlers also