PB_Truncate函数截取小数点引起的数字错误
參考PB幫助中Truncate函數的說明:
Using Truncate on a computed field????? A real number loaded into a floating point register (used for calculation) is represented as precisely as the binary storage will permit. For example, the real number displayed as 2.07 is actually stored as 2.0699999999999997.
Truncating such a number may not give the expected result. To avoid this problem, you can change the initial real data type to long, integer, or decimal, or you can append a constant in the truncate argument: Truncate (x + 0.0000001, n )
?
xg1=dec(left(mpart[8],4))
gonglv=truncate(xg1 / 100,2)
?
注意,這里的xg1如果是int類型的話,就會發生上面的錯誤,將其類型修改為DEC后,解決!
轉載于:https://www.cnblogs.com/wjonjon/archive/2010/07/14/1777023.html
《新程序員》:云原生和全面數字化實踐50位技術專家共同創作,文字、視頻、音頻交互閱讀總結
以上是生活随笔為你收集整理的PB_Truncate函数截取小数点引起的数字错误的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: ASCII,unicode, utf8
- 下一篇: 【原创】FPGA (Verilog/NI