Dealing with integers

[shaunfrost 9]

Does anyone have a simple function or demo on how to deal with numbers that need truncating? I have a virtual device that calculates a bunch of stuff, but I need to trim the number down to 2 decimal places... (its a LOT of decimal places at the moment!)

[Jamie mccrostie 82]

1 hour ago, shaunfrost said:

Does anyone have a simple function or demo on how to deal with numbers that need truncating? I have a virtual device that calculates a bunch of stuff, but I need to trim the number down to 2 decimal places... (its a LOT of decimal places at the moment!)

ROUNDIT

Rounds a number to a defined number of decimal places

This function takes one parameter - num (the number you want rounded) and idp (the number of decimal places)

so roundit(2.12348, 3) would result in 2.123 

Please login or register to see this code.

Thank Autofrank for this, its out of one of his tutorials.

[jgab 2,642]

If it's for formatting the number to a string for use in a VD label the builtin function string.format is quite useful;

Please login or register to see this code.

string.format have a lots of flags to control text/number formatting.

If it's the number you are after a lazy way would be tonumber(string.format("%.3f",2.12348)). Not as efficient as roundit though...

[shaunfrost 9]

7 hours ago, Jamie mccrostie said:

ROUNDIT

Rounds a number to a defined number of decimal places

This function takes one parameter - num (the number you want rounded) and idp (the number of decimal places)

so roundit(2.12348, 3) would result in 2.123 

Please login or register to see this code.

Thank Autofrank for this, its out of one of his tutorials.

 

Thanks to all.. worked like a treat