binary,floating-point,decimal,floating-point-precision,significant-digits

The precision is fixed, which is exactly 53 binary digits for double-precision (or 52 if we exclude the implicit leading 1). This comes out to about 15 decimal digits. The OP asked me to elaborate on why having exactly 53 binary digits means "about" 15 decimal digits. To understand this...

python,json,google-app-engine,decimal

It appears that the decimal.Decimal class is patched somewhere in the Google App Engine SDK (or the module is reloaded), and this is done between the MySQL conversion library importing decimal and you importing the same. Luckily, we can work around this by updating MySQL conversion table: from MySQLdb.constants import...

Yes, you can write: int x = 100 - 0x100 + 0100; That mixes decimal with hex and octal. The values are all converted to binary anyway before the calculation occurs (and the compiler will do the calculation in this example; it won't be evaluated at runtime). And any of...

It seems like your toolchain has a 32-bit int type. The maximum value representable in such a type is 231-1, or 2,147,483,647. As you can see, that's a 10-digit number. You'll need to use a different type that supports larger numbers if you want to use this kind of an...

Regex could be a solution since your numbers seem to be declared as strings. Code : <?php $re = "/(\d\.\d{2})(?!\d)/"; $array_input = array('1.00', '1', '1.000'); foreach($array_input as $row) { if(preg_match($re, $row, $matches) == 0) echo $row . " isn't a valid value with 2 decimals only. <br>"; else echo $row...

Try this code: bomba1.setText(new DecimalFormat("#.##").format(conta1)); ...

No need for looping. You can simply use the following formula: Math.ceil(-Math.log10(d)) Example: public static int firstSignificant(double d) { return (int) Math.ceil(-Math.log10(d)); } // Usage System.out.println(firstSignificant(0.123456)); // 1 System.out.println(firstSignificant(0.0012345)); // 3 System.out.println(firstSignificant(0.000012345)); // 5 A note on some corner cases: 0 has no first significant digit. For this case the...

I went with @tia and Dai's suggestion to use a Rational data type. I got tompazourek's Rationals NuGet package and used it like this: Rational a = 1; Rational b = 2; Rational c = 6; Rational d = 4; Rational e = 7; Rational f = 8; Rational g...

java,string,decimal,bigdecimal,scientific-notation

If you look at the toString() function of BigDecimal, it explains how the output is created: Next, an adjusted exponent is calculated; this is the negated scale, plus the number of characters in the converted unscaled value, less one. That is, -scale+(ulength-1), where ulength is the length of the absolute...

For user input, you usually want CultureInfo.CurrentCulture. The fact that you're using a locale that's not natural to you is not the usual user case - if the user has , as the decimal point in their whole system, they're probably used to using that, instead of your preferred .....

Use format bank to get 2 decimal displayed. More details about the format function in the documentation. If you want to change how the variables are displayed in the variable editor, have a look at this page of the documentation. ...

c,printf,decimal,octal,format-specifiers

011 is an octal value and its decimal equivalent is 9. Preceding integer literal with 0 indicates octal value. Use %o specifier in printf to print the value in octal.

php,mysql,casting,formatting,decimal

This should work : SELECT team , COUNT(*) as played , REPLACE(SUM(win), '0.', '.') as wins , SUM(loss) as lost , SUM(draw) as draws , SUM(SelfScore) as ptsfor , SUM(OpponentScore) as ptsagainst , SUM((win*2 + draw)- loss) as score , CAST(SUM(win + (draw/2))/SUM(win + loss + draw) as decimal(4,3)) as...

3.0 and 3 is the same number, so you can only check for a decimal point if you have string representations of numbers. If that's what you have, you can use the following: print "$_\n" for grep { !/\./ } @numbers; -or- print "$_\n" for grep { /^[0-9]+\z/ } @numbers;...

xslt,decimal,rounding,xslt-2.0,data-type-conversion

Assuming there is no schema involved, the typed value of the element InvoiceAmount is xs:untypedAtomic - which essentially means a string, but one that adapts itself to the context in which it is used. If you do an explicit conversion to xs:decimal, the untypedAtomic value "-62.84" will convert exactly. If...

You need to use the oveload of TextBoxFor that accepts a format string @Html.TextBoxFor(m => m.Price, "{0:0.00}", new { @class = "form-control", type = "text", id = "txtPrice", onkeypress = "return isNumberKey(event)"}) Side notes: Remove type="text". The html helper already adds this for you (add is there a reason why...

There are two ways to store decimal numbers: Fixed-Point: This is what you have defined in your SQL Server project. You defined decimal (19,4), which means 19 digits total and 4 digits after the decimal point. If you define such a type, then every number always has 4 digits after...

mov ebp, 3 ;ebp = 3(constant) mov eax, edx ;eax = edx(sum from previous calculation) mov ecx,3 ;load upper half of dividend with zero div ecx ;divide double register ecx eax by 3 It seems like your doing the division the wrong way! The code does not what the comments...

see: Decimal to binary conversion in c # it should be: void tobinary(int bin) { string binary = Convert.ToString(bin, 2);} ...

You pretty much answered your question yourself. Personally, I would not obsess so much about which method to use. If you method works now - even if it is undocumented - then it will most likely work in the future. And if a future update to .NET breaks your method...

Try this: NumberFormatInfo nfi = new NumberFormatInfo(); nfi.NumberDecimalSeparator = decimalMark; nfi.NumberDecimalDigits = decimalDigits; return value.ToString("F" + decimalDigits.ToString(), nfi); ...

The best option you have is to use Tryparse over Parse TryParse This overload differs from the Decimal.Parse(String) method by returning a Boolean value that indicates whether the parse operation succeeded instead of returning the parsed numeric value. It eliminates the need to use exception handling to test for a...

android,decimal,seekbar,android-seekbar

You should make your progress range from 0 to 200 and then multiply current progress with 0.1f when it changes. Cheers. SeekBar seek=(SeekBar) findViewById(R.id.seekBar1); seek.setProgress(5 * 10); seek.setOnSeekBarChangeListener(new OnSeekBarChangeListener() { @Override public void onStopTrackingTouch(SeekBar seekBar) { } @Override public void onStartTrackingTouch(SeekBar seekBar) { // TODO Auto-generated method stub } @Override...

datetime,matplotlib,formatting,decimal,ticker

Without a MWE its hard to know exactly, but I think you need to set the major_formatter. Assuming you have an axis object ax, you can use: from matplotlib.dates import DateFormatter, MinuteLocator ax.xaxis.set_major_locator( MinuteLocator(byminute=range(0,60,10)) ) ax.xaxis.set_major_formatter( DateFormatter('%H:%M') ) ...

You can use NumberFormat; the #-sign mean that decimals are only shown if they're non-zero. System.out.println("Value in " + new DecimalFormat("0.########").format(m)); ...

man, you are so lazy, but anyway, try this System.out.println(NumberFormat.getNumberInstance(Locale.US).format(1171906 )); ...

The only way to do it in a "single step" is to define a method, use the BitConverter: public static int ReverseByteOrder(int value) { byte[] bytes = BitConverter.GetBytes(value); Array.Reverse(bytes); return BitConverter.ToInt32(bytes, 0); } Usage: int value = 0x1480D604; //or parse from string int decValue = ReverseByteOrder(value); //decValue = 0x04D68014 IsLittleEndian...

javascript,decimal,number-formatting

this worked for me var value = 0.0038575; var loss = parseFloat(value).toFixed(3); ...

javascript,jquery,json,decimal

Use the javascript numToFixed function: var num = 5.56789; var n = num.toFixed(2); The result of n will be: 5.57 ...

wpf,textbox,format,decimal,custom-controls

I have written a custom control which will have dependency property called ActualText. Bind your value into that ActualText property and manipulated the Text property of the textbox during the gotfocus and lostfocus event. Also validated for decimal number in the PreviewTextInput event. refer the below code. class TextBoxEx:TextBox {...

javascript,function,decimal,fractions

the value of num is not updated, it is not passed by reference. function parseFractions(x) { var y = x.split(' '); if (y.length > 1) { var z = y[1].split('/'); var a = (+y[0] + (z[0] / z[1])); x = a; } else { z = y[0].split('/'); if (z.length >...

python,numpy,decimal,complex-numbers

You should look at the gmpy2 library. It provides fast arbitrary precision integer, real, and complex arithmetic.

You cant multiply or devise float and decimal.Decimal() types, what I would suggest is multiplying by Decimal('0.0023') and Decimal('0.5'): for a in range(0,size) et = Decimal('0.0023')*ralist[rows[a][2]] * ( Decimal('0.5')*(rows[a][3] + rows[a][4]) + 17.8 ) * ( rows[a][3] - rows[a][4])**(0.5) eto_values.insert(a,et) ...

Demo that got that result: class Program { static void Main(string[] args) { string unconverted = "äüö"; // is what you get when using File.ReadAllText(file) byte[] converted = Encoding.Unicode.GetBytes(unconverted); converted = converted.Where(x => x != 0).ToArray(); //skip bytes that are 0 foreach (var item in converted) { Console.WriteLine(item); } }...

excel,vba,decimal,strikethrough

For the sake of an answer: The Long data type contains integer numbers. per OP: solution was to change 'As Long' to 'As Single' I suppose I should add a little detail about Single Holds signed IEEE 32-bit (4-byte) single-precision floating-point numbers ranging in value from -3.4028235E+38 through -1.401298E-45 for...

It is not due to overflow you get the strange result. Doubles can easily hold numbers in the range you are showing. Try to print the result without setprecision. EDIT: After trying long double x = 100.1; cout << x << endl; I see that it doesn't work on my...

If you want to take the length of the string into account you have to tell it to the key function. We have to construct a key where the float value is considered first and the length of the string second. The easiest way to do this is to create...

c#,sql-server,ado.net,decimal,dbtype

The above is expected, if you have DECIMAL(2,8), you're saying a value no larger than 2 digits, and with 8 digits to the right of the decimal, which doesn't make sense. SQL Server would throw an error in this scenario... The scale must be less than or equal to the...

sql,sql-server,vb.net,integer,decimal

You're trying to access the value in the datatable cell with dt.Rows(0).Item(0).value.ToString("N2") but dt.Rows(0).Item(0) (or simply dt.Rows(0)(0)) already return the value as an object. You use value property to retrieve the value from a DataGridViewCell. The "right" way to do it would be: CDbl(dt.Rows(0)(0)).ToString("N2") ...

you can restrict the input using keyPress event like so: private void textBox1_KeyPress(object sender, KeyPressEventArgs e) { char c= e.KeyChar; if (!char.IsDigit(c) && !char.IsControl(c)) { e.Handled = true; } } if we want to extend our restriction condition to accept a certain character (for example ,) private void textBox1_KeyPress(object sender,...

I think if I don't use Round then this solution is fine. var valOne = 1.1234560M; // Decimal.Round(1.1234560M, 6); Don't round. var valTwo = 1.1234569M; // Decimal.Round(1.1234569M, 6); Don't round if (Math.Abs(valOne - valTwo) >= 0.000001M) // Six digits after decimal in epsilon { Console.WriteLine("Values differ"); } else { Console.WriteLine("Values...

java,android,decimal,decimalformat

You can use the "E" DecimalFormat formatting specifier to specify scientific notation, with a "0" on the end. DecimalFormat df = new DecimalFormat("#.##E0"); System.out.println(df.format(1.2345679E6)); This outputs: 1.23E6 ...

To avoid the confusion between int and float and to avoid the issues with arithmaic operations between the two types, I propose scan() the input as float/double. Use sprintf() to convert that as a string. Tokenize the string using strtok() based on the delimiter . (decimal point). First part (token)...

your best to use row 1 as a row to hold the maximum value for that column | A | B | C | --------------------------------- #1 | 200 | 400 | 800 | #2 | 1,1 | 2,1 | 3,1 | #3 |formula|formula|formula| then use the following formula =IF(A2+0,1>A$1,A2,A2+0,1) =IF(B2+0,1>B$1,B2,B2+0,1)...

java,decimal,decimalformat,place

Use String.format("%.1f", WAverage); Instead of df.format() No need for DecimalFormat...

nhibernate,oracle11g,fluent-nhibernate,decimal,buffer-overflow

So, what I have figured out is as follows:- Database column is float(126) which has more precision than supported by .NET hence, in some cases exception is thrown when .NET cannot handle the data (overflow). (Oracle number to C# decimal) When a formula is specified in fluent mapping, a column...

You have few options, other than fixing your code, or calling SET ARITH_ABORT ON for your batch. Even if you do call SET ARIGHT_ABORT ON, it will only raise an error in cases of overflow, and will not complain about loss of precision/scale (such as when the significant digits are...

The general method goes something like this: Work from right to left, you'll want to count the positions (starting with zero) and sum up the terms according to a the following formula: Say you're working in base x. Then, if you're at the ith position, and that digit is d,...

Just use 0.0 as format string instead of 0.00. As format strings for real numbers go, the number zeroes in front of the dot describe left padding, and the number of zeroes after dot describe the number of decimal places you want. If you want one decimal place, just say...

function trimDecimalZero($num, $delim = ',', $tsep = ' ') { @list($base, $decimals) = explode('.', rtrim(number_format((float) $num, 4, '.', $tsep), '0')); if (intval($decimals)) { return sprintf('%s%s%s', $base, $delim, strlen($decimals) < 2 ? $decimals .'0' : $decimals); } return sprintf('%s%s%02d', $base, $delim, $decimals); } $nums = [4.5000, 0.0050, 5.0000]; foreach ($nums as...

javascript,arrays,sorting,decimal

By default sorting is alphabetically. You need to pass a function to the sort method arr.sort(function(a, b){return a-b;}); ...

python,mysql,decimal,rounding,mysql-python

The number column in the Testing database table apparently has an integer data type. You can check the data type by querying EXPLAIN Testing. If it has an integer data type, the number value is coerced to an integer before it stored in the table. If you wish to store...

You could do something like this: function decimal($number){ $num = explode('.', $number); return $num[1]; } $decimal = decimal(10.5); echo $decimal; // should be 5 ...

Ok I got what I want :) new DecimalFormat("###,###.#").format(fIncome) ...

java,hex,decimal,decimalformat

That number is larger than an int (Integer.MAX_VALUE is 231 - 1 or 2147483647). You could use a long like System.out.println("Decimal value: " + Long.parseLong("A49D43C5", 16)); Output is Decimal value: 2761769925 ...

Do you need these numbers for future numeric operations? If not, then you should be able to store the data as a variable length character string so VARCHAR could work. here this website may answer your questions: http://dev.mysql.com/doc/refman/5.6/en/numeric-types.html...

There are some different ways to achieve this. First, it is possible with float_format and your locale, although the use is not so straightforward (but simple once you know it: the float_format argument should be a function that can be called): df.to_clipboard(float_format='{:n}'.format) A small illustration: In [97]: df = pd.DataFrame(np.random.randn(5,2),...

r,numbers,decimal,decimalformat

The commenters are right. It doesn't matter how long the list of numbers is. R can do the operation in one go. x <- c(25.01, 25.8, 25.4) x-15 [1] 10.01 10.80 10.40 If you do run into a situation with many wide-ranging numbers and need to literally keep what comes...

c++,binary,decimal,floating-point-precision,significant-digits

what is the most significant decimal digits precision that can be converted to binary and back to decimal without loss of significance? The most significant decimal digits precision that can be converted to binary and back to decimal without loss of significance (for single-precision floating-point numbers or 24-bits) is...

Easiest/Dirty way is to go to your PACKAGE/THEME folder and hunt down: app/design/frontend/base/default/template/catalog/product/price.phtml Then do str_replace( ',00', ',-', $fPrice ); A better way is to copy app/code/core/Mage/Core/Helper/Data.php Into app/code/local/Mage/Core/Helper/Data.php And do the str_replace there: line 153: return str_replace( ',00', ',-', Mage::app() ->getStore() ->formatPrice($price, $includeContainer) ); ...

c#,sharepoint-2010,decimal,currency,typeconverter

Convert.ToDouble() should work fine (or double.Parse()). The warning you see is related to trying to convert a DateTime to a decimal/double. If you look at the overloaded version that converts a string, I suspect you'll see no such warning. Try: //assumes input has been validated spli["Airfare"] = double.Parse(boxAirfare.Text); ...

perl,binary,type-conversion,decimal

You can use Math::BigInt. Please note, that input to these functions should be strings. use Math::BigInt; sub bin2dec { my $bin = shift; return Math::BigInt->new("0b$bin"); } sub dec2bin { my $dec = shift; my $i = Math::BigInt->new($dec); return substr($i->as_bin(), 2); } print "Dec: " . bin2dec("111110001011111001010101000010011001000010101111001110000000000000000000000000000000000000000000") . "\n"; print "Bin:...

you just need to do modulus and divide in similar order: int val = 7892936; while(val > 0){ int mod = val%256; print mod; val /= 256; } So the result will be: 200 111 120 ...

numbers,hex,decimal,autoit,definition

As per AutoIt documentation - Language Reference - Datatypes: In AutoIt there is only one datatype called a Variant. A variant can contain numeric or string data and decides how to use the data depending on the situation it is being used in. Issuing ConsoleWrite(0x00200000 & @LF) demonstrates stated behavior....

java,split,double,decimal,fractions

You shouldn't work with doubles as you are losing precision and this can lead to serious errors. But in the case of 1.0001 the problem is that: Double.toString(1.0001) == "1.0E-4" Then you try to parse "0E-4" and you get 0 instead of 1. You could do the following if you...

try this: use strict; use warnings; my $limit = 0.5; my $result = ""; my @inpt = qw(1003.3965 1036.4392 1037.3843 1045.4459 1101.4259 1107.4253 1118.3928 1191.4904 1320.4855 1420.6291 1440.6921 1562.6698 1742.7587 2084.9137 2248.8761 2337.8865 2628.9931); my @outpt = qw(1191.6017 1101.5336 2629.2865 1742.9336 1036.5726 2338.1574 2249.1057 1440.8222 1440.2074); foreach my $s (@outpt){...

Use CultureInfo. If you are sure that using decimal dot is a standard in your country then use your country's CultureInfo. You can simply check if it is. // Change kk-KZ to your culture code decimal dec = decimal.Parse(str, new CultureInfo("kk-KZ")); Or you can use InvariantCulture and NumberStyles.Any: decimal dec...

Pass the precision/decimals variable to str.format: decimals = int(input("What amount of decimals do you want to see of the golden ratio?")) ratio = gr(41)/gr(40) print("{:.{}f}".format(ratio,decimals)) Example: In [3]: decimals = int(input("What amount of decimals do you want to see of the golden ratio?")) What amount of decimals do you want...

A decimal(4,3) has 4 digits in total, with 3 after the decimal point. See decimal and numeric in the SQL Server online docs, particularly around precision and scale. It will hold values from -9.999 to +9.999. To support the value 10 you could use a decimal(5,3)....

You can use the modulus operation. With integers this is the % operator, with floats you can use std::fmod // \brief Calculates the sum of two angles // \param[in] a First angle [degrees] // \param[in] b Second angle [degrees] // \return Sum of two angles [degrees, [0, 360)] double AddAngles(double...

You can match ASCII number codes by using \x to match escaped hexadecimal codes, e.g: \x02 should match STX...

You can round a decimal to one fractional digit and then compare them. if (Decimal.Round(d1,1) == Decimal.Round(d2,1)) Console.WriteLine("Close enough."); And, if rounding (with default midpoint handling) is not what you want, Decimal types can also be used with all the other options, like those I covered in this earlier answer....

print 'Your total amount due is ${:.2f}'.format(totalCost) And: print 'Your total amount due is $%.2f' % totalCost both print to two decimal places. In [3]: 'Your total amount due is $%.2f' % 1.2234 Out[3]: 'Your total amount due is $1.22' In [4]: 'Your total amount due is ${:.2f}'.format(1.2234) Out[4]: 'Your...

entity-framework,decimal,breeze,scale

This is an EF problem. The default scale for decimal types in EF is 2. You have to tell EF what the scale is for your column. If you only need 4 decimal places, you could use a shortcut by saying it is a Money type: [Column(TypeName = "Money")] public...

100/65 is an integer division. What you need is double d = (100d / 65) % 1; This will give you 0.53846153846153855...

c,string,recursion,type-conversion,decimal

I see that you have implemented it correctly using global variables. This works, but here is an idea on how to avoid global variables. A pretty standard practice is adding parameters to your recursive function: double finalValue_recursive(char *string, int flag1, int data2) { ... } Then you wrap your recursive...

well multiplying 2.5 with 2.5 still results in 6.25 ;-) at least my calculator tells me that. What you probably wanted to do was echo $sale_limit_gen['value'] * $sale_limit_ale['value']; instead of echo $sale_limit_ale['value'] * $sale_limit_ale['value']; so its a matter of a simple typo....

You can use this code: sb.append("\"2.0\""); I often use even more verbose, to force text format: sb.append("=\"2.0\""); (see http://stackoverflow.com/a/165052/476791 ) To verify, that this solution works open notepad, enter ="2.0", save it as Test.csv on your Desktop and double click it....

r,csv,decimal,decimalformat,decimal-point

#some sample data write.csv(data.frame(a=c("1.234,56","1.234,56"), b=c("1.234,56","1.234,56")), "test.csv",row.names=FALSE,quote=TRUE) #define your own numeric class setClass('myNum') #define conversion setAs("character","myNum", function(from) as.numeric(gsub(",","\\.",gsub("\\.","",from)))) #read data with custom colClasses read_data=read.csv("test.csv",stringsAsFactors=FALSE,colClasses=c("myNum","myNum")) #let's try whether this is really a numeric read_data[1,1]*2 #[1] 2469.12 ...

Math.round(x) //x being your number you wish to 'round JSFiddle : http://jsfiddle.net/j37m78a4/2/ I have commented out the code that provided an answer with decimal point values. Uncomment that and comment out the line below it to see what the original answer was. :) If you don't know where to put...

Here is a proof of concept using 2 rather different data frames: DF1 <- data.frame(x = rnorm(10), person = rep(LETTERS[1:2], 5)) DF2 <- data.frame(y = 1:10L, result = rep(LETTERS[3:4], 5), alt = rep(letters[3:4], 5)) write.table(DF1, file = "example.csv", sep = ",") write.table(DF2, file = "example.csv", sep = ",", append =...

python,decimal,scale,maya,arbitrary-precision

Maya's native precision is usually based on 32-bit floats (for most linear distances and general purpose math) and 64-bit doubles (used mainly for angular values). Python floats are basically doubles, so they should be 'lossless' for all practical purpose although I've never tried to check that the python and C++...

ruby,paypal,floating-point,decimal,money

I would use Ruby's built-in BigDecimal to get more accurate rounding: require 'big_decimal' amount = BigDecimal.new('2.12') tax_rate = 0.2 (amount * tax_rate).round(2).to_f #=> 0.42 In this case, however, I think it's your own rounding method that is causing the problem because: amount = 2.12 tax_rate = 0.2 (amount * tax_rate).round(2)...

python,decimal,fixed-point,exponent

From the documentation: Q. Once I have valid two place inputs, how do I maintain that invariant throughout an application? A. Some operations like addition, subtraction, and multiplication by an integer will automatically preserve fixed point. Others operations, like division and non-integer multiplication, will change the number of decimal places...

python,powershell,floating-point,decimal

Change print "You are %d years old." % age to print "You are %g years old." % age By the way, eval is not necessary there, the floatconversion suffices....

I would say this is to be expected since you have a power computation of the kind 10**x here, which in the case of decimal should mean checking for precision after every step. Consider the example below, where it is clearly visible that the decimal power operation is very costly:...

python,decimal,rounding,currency,babel

If you follow the code through to apply you can see a reference to a bankersround procedure, which tells us what we need to know. Babel is using the bankers round method of rounding, which rounds to the nearest even number on 5. So, .245 and .235 round to .24...

python,string,decimal,scientific-notation

If you want to reproduce your decimal, just construct it from the string: >>> d=decimal.Decimal("0E-1000") >>> str(d) '0E-1000' I believe the difference is the difference between 0 and 0.00000 (or in your case 1000 zeros), which is the accuracy. This could be significant in scientific situations, where rounding etc. is...

python,while-loop,binary,floating-point,decimal

I don't think this code is written all that well, but here's a rough idea. The first while loop: while ((2**p)*x)%1 != 0: ... is figuring out how many places in binary to the right of the decimal point will the result be. To use a familiar analogy let's work...

There are two ways you could modify your code to do what you want. The first, would be to change the printf to print the character representation rather than the integer representation of the value. printf("%c", str[i]); Will print "0" given the in put of the string "0" from the...