BEWARE Crossing the OFFER with your BID on the BitShares DEX, or you may OVERPAY and/or BUY GREATER SIZE than you EXPECT!
While I think @abit's approach of flagging down this post and calling it "FUD" is an outrageous abuse of his "weight" on the steemit platform, perhaps I shouldn't be too surprised. In China, the government often believe censorship is the ideal solution for "keeping the masses inline". I suppose @abit subscribes to this point of view as well, despite the fact that it goes against everything that STEEMIT is supposed to stand for.
As such, it turns out @abit, @pc, and myself are both right and wrong about what's going on here. However, I think it would have been much more productive and efficient to work together on solving the problem, without all the "drama" @abit chose to bring to the table.
Once I understood what was really going on, I was also able to easily reproduce this behavior in the testnet as well. Upon @abit repeatedly stressing "rounding errors", I decided to experiment a bit more to see if perhaps I was underestimating how large these discrepancies could be, even with seemingly "reasonable" pairings. It just didn't make sense to me how it would apply in this situation. But if division is being used non-optimally, it's true that rounding errors can compound awfully fast. As a simple example, 0.01 is just one penny. But if you invert, 1/0.01 = 100, and 1/0.02 = 50.
But why would any of this type of behavior matter here? We have a sitting offer. Yet, somehow we're seeing fills coming in at much higher prices. How is this a rounding error???
I'll just jump right in and tell you.
The "problem", at least for the community to decide if they consider it a "problem", is that you're not just "buying", you're "Buying at least". THAT's where the "rounding errors" are compounding.
Take the following example. Let's say there's an offer of 4.50, and by accident, I bid 4.60 for 10 instead of 4.50 for 10. Most of us would think no big deal, we'll just be getting 10 at 4.50 each. That's NOT how bitshares currently works. What you'll get instead is 10.0178 at 4.50.
And if that 4.50 price didn't "round" evenly, the figure may have been higher, in some cases much higher. But in most cases @abit and @pc are probably right. Hopefully it should balance out relatively evenly, except for certain "fringe" cases...
Such as buying 0.01 at 8.17 instead of 8.16138. That seems to be a problem. Buying 0.0002 HERO at 925 versus the 919 offer also seems to be a problem, because bitshares cannot properly quantify the size adjustment versus the current offer.
Now let's see a really extreme example of this play out on the TESTNET with what should be just a "small" order... What if I want to buy 1.5 INTELLIGUYPIZZA at 4.50 TEST, but by accident I entered 345 instead. Yes, I'm aware of the "GUI warning". But that's not the point here. The offer is 4.50, and there's plenty available. So, no problem... instead, I'll just get my 1.5 at 4.50 instead of 345, right?
WRONG!
Here's what you'll really be getting... bought 113 INTELLIGUYPIZZA at 4.50 TEST/INTELLIGUYPIZZA. If the size versus price didn't quite match (hence, the "rounding error"), the execution may have also taken place at a slightly higher price than 4.50 as well.
But, of course, you also just bought 113 instead of the 1.5 you were expecting. Personally, I don't think this is the correct way it should be handled. I would have expected 1.5 at 4.50, and that's it.
But what about those crazy WHALESHARE buys?!
Now, while all this seems to answer most cases that I came across, it DID NOT seem to properly address the WHALESHARE_BTS buy 40% higher than the inside offer. UNLESS, of course, this "size adjustment" is targeting the NON-DECIMAL LEG for adjustment! Well then, that should be easily testable in the TESTNET. So I created a new NON-DECIMAL asset called WHALEHOLE. And this effort did not disappoint. I'll just give you the screen shots, and let you make of it what you will...
Once again, I may not have gotten the exact cause quite correct the first time around, but I stated as much, and I very much appreciate those who supported my position throughout this "experience". My only goal here was to inform users of a critical issue, and to help make bitshares safer and more "user friendly" for all those using the platform. It may have its quirks, but in my opinion, it's still the best decentralized trading solution that currently exists. Had @abit chosen a different path of working together, instead of immediately going for the jugular, perhaps we could have worked this out much more productively and with fewer hard feelings.
But, at least I'm happy that we now all better understand what exactly is going on. If this is the way the community believes the bitshares network should function, great. If not, well, perhaps there's a "discussion" to be had after all...
Well, this one's a real doozy, even next to the other BitShares "feature" I wrote about a few weeks back, suggesting to be extra careful when trading 2 non-decimal tokens against each other on the BitShares DEX, lest you end up pay much more than you expected for your tokens.
And while you should always be careful to double-check (triple-check?) the orders you enter, be extra careful not to place a bid that crosses above the offer (ie. don't try "sweeping" the offers). With most trading systems, you would naturally expect your crossing bid to pair off with the lowest offer. However, depending on the scenario, the bitshares network may simply fill your order at the price entered, no matter how far above the offer it's "fat fingered"! 😱
This "journey" began when I noticed some trades that had executed straight through a WALL of sell orders. Those I had first mentioned it to in the BitShares / Beyond Bitcoin Hangouts seemed highly skeptical it could even happen, and offered suggestions such as "perhaps all the offers were momentarily pulled". cryptofresh.com is a fantastic tool for exploring the bitshares blockchain and tracking down orders, executions, counterparties, etc. However, closer analysis of these trades revealed nothing of the sort taking place. Most of the offers were simply standing orders, sitting on the book for hours or even days. And if someone stepped up with a BID that overshot the OFFER, they would immediately be filled at the "fat finger" price!
Frankly, I'm surprised no one has yet attempted to correct this issue, especially given that I found references circling the problem going back to 2015, when BitShares 2.0 was first "unleashed". I was especially confused when all the overpriced crossing orders I placed on the bitshares testnet all crossed perfectly, exactly as they were supposed to. What could possibly be causing my real-world experience (and many unfortunate others) to be so different from the testnet? Was it a GUI issue, or a blockchain issue?
One of my earlier thoughts was that perhaps the GUI was responsible for making sure an order doesn't overpay. However, that did not appear to be the case either. Before reaching that conclusion however, I placed over-priced orders from both the web GUI and the app GUI, with identical results...
And to be sure, I used Telerik's excellent program Fiddler (the free web debugging proxy) to monitor the websocket conversation and figure out what's going on. Low and behold, I was able to confirm that the overpay was being sent out to the server, and we were receiving the appropriate price adjustment after all! So it's definitely a blockchain issue...
outgoing: {"method":"call","params":[3,"broadcast_transaction_with_callback",[181,{"ref_block_num":39315,"ref_block_prefix":1899533150,"expiration":"2017-07-14T22:05:39","operations":[[1,{"fee":{"amount":"100","asset_id":"1.3.0"},"seller":"1.2.3252","amount_to_sell":{"amount":"540000","asset_id":"1.3.279"},"min_to_receive":{"amount":"100","asset_id":"1.3.280"},"expiration":"2022-07-14T22:05:17","fill_or_kill":false,"extensions":[]}]],"extensions":[],"signatures":["20385f468195f7c5270984b38ed6f3855ee257d13b3914b636f690c32cb50661785ed539e60068a8bdbed8cc6fea0d96e3d1cfa134fbca0bcf26dec30ae17c3849"]}]],"id":181}
incoming: {"id":181,"jsonrpc":"2.0","result":null}
incoming: {"method":"notice","params":[50,[[[[4,{"fee":{"amount":2,"asset_id":"1.3.280"},"order_id":"1.7.69096","account_id":"1.2.3252","pays":{"amount":540000,"asset_id":"1.3.279"},"receives":{"amount":108,"asset_id":"1.3.280"}}],[0,{}]],[[4,{"fee":{"amount":0,"asset_id":"1.3.279"},"order_id":"1.7.69074","account_id":"1.2.3246","pays":{"amount":108,"asset_id":"1.3.280"},"receives":{"amount":540000,"asset_id":"1.3.279"}}],[0,{}]]]]]}
To summarize that bit of gobbly-gook, what you're seeing here is that I tried overpaying with a bid of 54 versus a resting offer at 50. The outgoing order shows "amount:540000" versus "min_to_receive:100".
540000/108 = 5000 / 100 (for the decimal) = 50, or exactly what it should be!
But what the heck, I'll give you just one more example where I overpaid when buying some HERO versus BTS on the live bitshares network. Isn't this fun?! Well, just you wait... it's about to get a whole lot FUNNER! lol
Offer sitting at 919 bts, but I had the distinct "pleasure" of BUYING at 925 bts!
GREAT, you made your point... what's really going on here?
My original thought was that the problem was margin-call related. As described at the top of this post, I was incorrect in that assessment, so I have removed this section of the post and defer you back to my revised explanation.
The "Journey" to Solving the PROBLEM...
To track down the "source" of this problem, I chose to load up the bitshares source code into Visual C++ and step through to see exactly what was going on. After installing VC 2013 and VC 2015 (which I had managed to avoid until now), and attempting to follow multiple step-by-steps to get it running, well, needless to say, I just couldn't figure out what I was missing to get it going...
Error C2027 use of undefined type 'std::_Get_function_impl<_Fty>' graphene_app C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\functional 472
Regardless, some of you may recall my post a few months back, "EOS: What's Your BEST Bang-for-the-Buck Cloud Provider?". And after wasting way too much time on Visual Studio, I decided this may be a great opportunity to give scaleway.com a try. Worst case it sucks, best case, well who knows... but how could I really go wrong with their 3 euro / month VM?! I got 2 x86 64-bit Intel Atom cores at 2.4ghz, 2gb memory, 50gb SSD, and 200Mbit unmetered bandwidth. I setup an Ubuntu 16.04 droplet with an xfce desktop (accessed via x2go), and frankly, it's been rock solid ever since! I've gotta say, for 3 euro a month, you really can't go wrong. And hey, if this post does well enough, perhaps I can "splurge" for their 10 euro/month version with 8gb and 6 cores! 😁 (just kidding...)
The biggest problem was that the codebase did take a long time to compile, but that always seems to be the case with large C++ projects, and while a bigger droplet would probably help somewhat, many others running witness servers, steem nodes, etc, have also complained about long build times, despite deploying much more powerful droplets.
Nonetheless, I finally got it all going... running and debugging bitshares-core via the excellent CodeLite IDE. I first tried using the CodeBlocks IDE, but kept hitting a wall. Perhaps I was just missing some simple setting somewhere, but with CodeLite, I was pretty much up and running my first attempt!
EOS too?!
As an added bonus, and especially timely as @dan just announced the "EOS Development Sneak Peek for Very Early Developers", I even managed to get an EOS testnet up and running on the same droplet!
There were a few bumps with the current build at the time. I also had to create a swapfile, because 2gb didn't cut it for EOS to compile. However, after creating a 6gb swapfile, and once the EOS team corrected the issues with the docker configuration, it was more or less smooth sailing from there. For those who would like to build an EOS testnet without using docker, @clayop wrote a nice "step-by-step" article describing how do that as well.
For those interested, you may want to read this github thread describing some of the issues that came up. Needless to say, I was now also "mining" EOS testnet blocks! :D
Finally...
It's hard to say how long it may be before this problem is corrected, or even if it is corrected. Some may see this as a "feature", as opposed to an "unexpected pattern of behavior" (certainly those seeking to take advantage of this "loophole"). As such, I would posit that confidence in the system and the way it is expected to perform is critical to attracting more liquidity, and supporting the longer-term growth objectives of the platform.
Regardless, it's always prudent to double-check that your limit prices do not crosses the market, or else you may end up overpaying, in some cases quite dramatically! Much of this is bleeding-edge technology, and there are bound to be issues that still arise here and there. Just take the recent bitshares downtime a week or so back. @dan quickly solved the problem, despite all the work he's doing on EOS. The network came back up, and the world moved on. "Worry and panic" over the "future of the network" was mostly unfounded.
Over the years, I've witnessed countless times when the Chicago Mercantile Exchange, the New York Stock Exchange, NASDAQ, and other mega exchanges exhibited serious downtime as well, not to mention some really whopper order-matching foul-ups. As always, "Murphy's Law" universally applies... If something can go wrong, it will go wrong. Sh!t happens...
Just be aware, and stay safe out there my friends! 🙂
UPDATE: Apparently, my post has caused a bit of a stir in the bitshares community, and I'm glad at least I got the discussion started, and that the issue is now being looked into more seriously. For those interested in following:
BitSharesTalk Thread: https://bitsharestalk.org/index.php/topic,24715.0/all.html
GitHub Issue: https://github.com/bitshares/bitshares-core/issues/338
My first impression here is that this is not a bug. It may not be what you expected, but AFAICS you did receive the payment you requested (and even a little more).
The blockchain is acting somewhat aggressively wrt matching margin calls. That's OK IMO, because closing undercollateralized positions is important for the health of the bitasset.
But this is nothing about matching undercollateralized positions, as I believe a fat-finger order is simply falling into that category by accident (I'd be quite surprised if it was by design). And it is certainly NOT an "expected pattern of behavior". But, at least if more people are aware of the problem, they will be less likely to make such a mistake.
However, let me ask you this. If by accident you entered a BUY order for 8 BitUSD at 1,000,000 BTS instead of 1,000,000 BitUSD at 8 BTS, would you feel okay with your irreversible 8 BitUSD at 1,000,000 BTS each, despite the fact they were clearly available for under 8 BTS each? Perhaps that's something to think about... I suppose the bitshares network could always pull an "ethereum-type solution" and "bust" the trade if it's sizable enough. But somehow (to me at least) that doesn't quite seem the best way to go.
Fat fingers are unforgivable. You won't expect a refund if you placed a wrong order in other exchanges.
Margin calls are only one direction, they can only be orders to sell BTS for bitUSD. Your new order to sell 1M BTS for 8 bitUSD will compete with them, but not match with them. Whether your order will get filled or not, at what price, depends on the orders on the opposite side, say, who wants to buy your BTS, and at what price.
At present there is no margin call on the book of BTS/bitUSD pair. You will get best offers no matter what price you placed.
my response above addresses all these points as well
This is a good example, but also even having some lower asks come in while you're entering your bid or something like that and still being filled at a higher price is unacceptable. Other trading platforms don't function like this, so why would you intentionally choose to utilize a platform that's designed to occasionally fuck you?
This post has been upvoted and resteemed in the name of free speech. I may not agree with what you say, but I agree with your right to say it.
I think you are doing bitshares huge service here. Shame on the flaggers
Good find! I'm happy to see that you are working on a solution to Improve the trading platform!!
thank you @willybobob, been working hard on this issue, so hopefully something will come of it as part of an upcoming hard-fork! :)
Your brain.. has me speechless right now.. how tf you figured that out.. I've been speculating for years and just "recognize" when I'm getting "worked over" on my trades but I can't imagine being able to actually DISECT what is going on beneath it all..
thank you for the kind words @avva. I'm hoping I got it mostly right. It's possible I'm off on parts of my analysis, and time will tell more how "on the money" it is. Regardless, the issue is very real, and more people must be made aware of it for the good of the ecosystem.
Geez, that is definitely noteworthy. Yes, you would expect to just get the best offer or bid on the sheet regardless. I will make sure not to be loose with my order prices going forward. Thanks again for another good heads up man!
yeah, at least on the stock market if you make that mistake, they can hopefully bust the trade! but of course, that generally would only happen in the first place if there's no counter-party to offset through, and not with an order wall staring you in the face!
Very informative, as always @alexpmorris. Thanks for taking the time to put all of your findings together.
Job well done, sir! ; )
thank you for the kind words ms @michellectv!
Glad to see you and abit were able to work things out and come to an understanding.
Have you transcribed this into a bug report on gitub? https://github.com/bitshares/bitshares-core/issues
thanks for asking @testlump, opened the following issue after I posted this: https://github.com/bitshares/bitshares-core/issues/338