Improving Steem's Hard Fork Process: Where's The Documentation? Decentralisation in Theory Only?
I wrote a reply earlier to @reggaemuffin's post on proposals for improved requirements for future hard forks of Steem - some good points are made in there. In my reply I related my experiences working for a software development company that made software for investment banks (over 15 years ago) - I learned a lot there regarding solidly reliable methods of development and teamwork (and also during my previous degree training in software engineering), yet most of it appears to be rarely used by the majority of software projects I see in the public eye today. Here's an expanded version of that reply.
Structured Design / Development is More than Just a Formality - Teams Need It!
In software development, types of structured / formal design and development are used in general for creation of systems where failure is not an option. These approaches include a high focus on production of documentation, diagrams and careful modelling/recording of the current system and any changes to it. This allows precise monitoring and management of changes and also ensures that failures can be dissected both reliably and independently to improve future performance. In short, these methods may seem in opposition to the idea of 'anarchy' and decentralisation, but in reality they are just a necessary part of the process of reliably making reliable software. At the very least they allow outsiders and short term contractors to get involved to help without having to spend weeks trawling through code and interrupting the core team with questions.
If we want to have a team working together remotely (over github and the web), then at the very least we need a coherent way for the team to grow through it's own effort and learning, based on good documentation. There are a wealth of smart and qualified people online and no doubt using Steem too - but if they can't find a way into the technicalities of the project then they can't really provide their expertise. What better way to use Steem's reward capabilities than for improving Steem? Yet, even Utopian - Steem's method for rewarding open source code development - doesn't even have the Steem projects Whitelisted, so we can't use it!
The problem I see over and over with the github / decentralised approach is that pretty much no-one produces proper design documents and technical writing (steem code doesn't even have full commenting). This means that the 'decentralisation' is highly limited since most people won't get involved as they can't gain mental overview of the project, plus it also means that it therefore isn't so much 'decentralisation' as it is 'centralisation with the convenience of public file storage for the project'.
I have pretty much never seen such formal methods used anywhere outside of military/gov/banking/science software areas (large corporate entities) - mainly because of the time and cost involved. Since most of the coding I am interested in takes place in open source now and on github.com and since these projects are largely non formal, it has been a long time since I saw ANYTHING approaching this level of design work on any modern project (although I don't spend my days looking at a large number of projects).
Don't Let The Old Systems Eat The Evolutionary/Revolutionary Ideas
Since cryptocurrency aims to literally topple the prevailing criminal banking systems - it is important that those involved don't take lightly the value of formal documentation and best practices for testing and teamwork. I can see a situation looming where the large corporate teams use their experience in such things to introduce their own version of crypto to the world and take over, as if this 'revolution' never happened. It's great to think freely and create new ideas such as Steem, but if it isn't backed up by a solid foundation, then it will be replaced.
Practical Steps For Improvement
It is not wrong to take the useful approaches from the 'old world' and use their best parts to go forward into a better world. I strongly urge that those involved take steps to increase:
- transparency
- formalisation of documents and designs
- visibility of documents/designs and roadmap
- potential for collaboration
- community involvement in testing
The funds available to Steemit inc. and even the top 20 witnesses make hiring technical authors, testers and the appropriate support staff easily within reach - yet I don't see any evidence of it being done. I can see that the Steemit job listing page has several technical vacancies, but none for any of the other architecture, testing, QA, technical writing or other support roles that are vital to success. Why is that?
My Own Personal Steps To Help
As a witness for Steem and as an avid user of Steem, I'd like to be able to help out more - but my hands are partially tied by the points already raised. Some of the top 20 witnesses apparently work together to test and improve the code to some extent, but they discuss this in a private chat room - so I don't really know the extent of it. Additionally, the code for Steem is largely in C++, which is a language that I personally haven't used for many years - so it would be a steep learning curve for me to become a dev for Steem right now. On top of all of this, I have bills to pay and am not earning enough from Steem to cover them.. So, I either need to get more witness votes to get more reward, to dedicate more time to the project - or I have to limit my help substantially.
At the very least I intend to find some time to learn to help to test future Hard Forks - as yet I don't know how practical that is for me though, without some financial help for my time.
I would be happy to help with diagramming and technical writing to some extent too - though I would need to have much greater access to the work that is being undertaken and the people involved than I do currently.
Anyway, some thoughts to chew on for you all!
Wishing you well,
Ura Soul
Vote @ura-soul for Steem Witness!
(Witnesses are the computer servers that run the Steem Blockchain.
Without witnesses there is no Steem, Steemit, DTube, Utopian or
Busy... You can really help Steem by making your 30 witness votes count!)
This is a great point, coming from the oil and gas project management world, where if you make one mistake a whole plant can explode, and people can die, I am used to working in this professional way, with procedures, and sign offs before executing things, along with meticulous planning. it ensures we get things RIGHT FIRST TIME. This way of thinking MUST be implemented into the Dev world of blockchain, as the current way of working that i see in blockchain tech will make easy targets of any oraganisation with just the slightest most basic methods or oragnisation, discipline and professionalism
I agree, yes. It's not necessarily a fun way to work - but it gets things done right first time, as you say. In two years of supporting numerous international banks and other institutions, I think we only had one minor issue that affected data in a client's system and afaik, no down time. It's only possible with high organisation and careful monitoring, planning and checks/balances.
So – here's the thing.
You cannot have "distributed anarchic development" and "organized and structured development" at the same time. It's literally impossible. It takes a strong managerial degree of oversight and really significant resources dumped into rewarding the people that do the least pleasant grunt job in software development (documentation) to make sure it happens. That's why projects in banking, oil and gas, and the other panoply of five-nines/high-availability/high responsibility engineering cost out the ass. Someone has to get paid to do work that nobody wants to do, and someone has to get paid to check all of that work that nobody wants to do – because nobody wants to do it.
Moreover, those things are expensive and time-consuming but organizations with strong central planning architectures execute on them because the consequences of their failures are even more expensive and their successes are ridiculously valuable.
None of these things are true with systems with anarchic, distributed development. If they become true, a central managing body steps in who has the money to invest in the stuff that nobody wants to do in order to reap the benefits of a lucrative success. And that is literally the best you can hope for.
If you're designing a system that requires delicate, highly documented, deeply reviewed software in order to function in a way that it's consistent enough to be trusted.
And that's really the point and the problem.
If we look at functional distributed network/communications systems, they are designed to be inherently resistant to the kind of failures that we see in the majority of blockchain designs including steem. They are designed to be robust in the face of nodes which are explicitly out of revision. Most of them are designed around a protocol specification with the intent that there be no "witnesses/miners," but instead individual participants are responsible for their part. And in the main, they can get away with that kind of anarchic, truly decentralized system because it's a system that's intended to be as minimal as possible in order to provide an attack surface for as few problems as possible.
That is not the case for the design of the steem blockchain. Quite the opposite.
So we have to know up front that the steem blockchain is not a good candidate for what we would think of as proper five-nines software design or software management. We have to know up front that everyone outside of Steem Inc. is effectively a volunteer, and volunteers hate to do boring work – especially volunteer engineers. We have to know up front that Steem Inc. is not interested in doing five-nines software development because they have had vast amounts of opportunity over the last two years to engage in it, and have actively shied away from it.
So what's the solution?
Well, we can continue to depend on the witnesses to theoretically advocate for good development methodologies and hope that they care about such things at some level, at some point enough that they will sink their own money into some sort of management organization – but I wouldn't hold my breath. And beyond that, not a damn thing.
Which is also about what we can expect in terms of change when it comes to how the system is managed.
That's really the bottom line.
I'm not sure about that. Structured development really just involves a set of stages that include deliverables that perform functions within the development process. Some deliverables are documents and some are code. The introduction of documentation into the process does not require 'rulers' in the sense of being something that makes anarchy impossible and it isn't prevented by the process being distributed geographically since we have the ability to share documents in realtime (plus communicated with audio/video.. We aren't yet able to hold hands remotely, but that isn't really a necessity here anyway).
All that is needed is for Steemit inc. to pay someone to do technical authoring and to work with the designers/coders. Usually it is ideal for such a person to also be experienced in system architecture and coding, but it isn't 100% essential. Steemit inc. already has a development team and a manager. The team works together, according to a plan and makes a product. It is true that to keep the process coherent requires a cohesion and the presence of a strong guiding force - but that does not need to be based in a top-down control model, it can actually just be a shared vision of the need for things to be a certain way. I have worked on a large open source social software project for a few years and while they didn't publish full documentation of the kind I described in my post - they do publish full commenting and do stick to rigorous requirements for those to be created for all new code. If code doesn't include comments that meet standards, it doesn't pass testing and get into the builds.
Having a true collaborative culture requires the understanding of the benefit to do so, but perhaps the difference is that in the other project I referred to, there was no direct financial element involved giving people a sense of 'losing out' if more people get involved.
In some cases that might be true, but the kind of documentation I am describing is actually a standard part of a structured design process, it is not 'an annoying addition' but rather is actually a part that forms the foundation of the rest of the work. In such situations there are many people who greatly enjoy producing such documents. This is similar to architects designing a house on paper - it might be time consuming work, but many love it all the same. Does a builder love making such documents? Probably not and probably has no training at doing it. Does a pure coder love making such documents? Again, many seem not to really know fully how to do it and so are not attracted to it.
Part of the issue with Steem, that many people are ignoring, is that the success of the project is not just about technicalities - it is about PUBLIC IMAGE. The value of tokens largely revolves around perception, hype and prediction (as well as actual value and utility based on tangible things). While a few days of difficulty in using the system might annoy people who use the system, in itself it doesn't equate to the kind of damage that might be caused by a system failing that is designed to measure earth movements to protect oil drilling systems, for example. However, not only does the image of the system get hugely hurt by such downtime (lowering the value of the system in some people's eyes) - it also infers that the use of Steem as an actual method of exchange, rather than just a social blogging platform, is not really as great an idea as it might otherwise seem. No-one is going to die through not blogging for a few days, but we lived in a world where Steem was a primary currency used for daily exchanges and we suddenly couldn't use it, then real problems could arise.
Anarchic dynamics have the potential to result in greater productivity, balance and effectiveness - imo - provided that those involved understand how to achieve that without relying on being controlled by others. In other words, true self empowerment and self determination - when used to create effective teams that work together through agreement to use best practice - can produce teams that are capable of employing such structured approaches simply because the approaches work well and those involved value them.
I appreciate that maybe the majority of those involved simply have no desire to work this way, though they have not said so and I for one don't really know the truth of that. I am just putting forward a reality that I think deserves to be considered.
I agree 1000% with you on this, @ura-soul
Thanks for speaking out... and for sharing your unusual mathematical achievement ;)
Thank you for doing the things I expect from a witness.
In the Navy as an electronics tech, I learned a 7-step troubleshooting procedure. Step 7 was to figure out what could be done in the future to prevent the original problem.
You are welcome. I think the suggestions here are mostly useful even when there is no problem, since they are also ideas that help the project grow even in good times.
I can understand management wanting to tighten up the scope of the project in order to avoid 'function creep' and challenges in managing communication and teamwork, but these too will be helped by improved and published design docs and commenting etc.
My take on this valid concern is that I would give all these boys in blockchain a bit time. Even we know Steemit Inc is heavily driving so it is a bit not really decentralised we should focus on the de-centralized young models we all are part of. I am a non tech guy and usually see all only from a marketing perspective. Sure it would have been good to do longer stress tests here - but at least it was communicated there would be some tech issues coming they can not foresee. A lot people complain right now about the implementation, it could have gone more smooth, fair enough but many of the guys come up with comparison to totally centralised models like banks. I see major trend here to go into a strange route of back to past - also solutions like steem-ua are totally centralised and not transparent. Take a look at EOS and what struggles they had, BTC chain had massive bugs - as long as in 5 days as anticipated all works well - fine to me.
My aim is in no way to go 'backwards' - it is just to affirm that some things are done a certain way in the 'old' systems for very good reasons - because they work effectively with minimal need for extra overheads.
It is totally understandable for a new company to be limited in terms of it's team and overstretched, perhaps - but Steemit has a lot of money available, as far as I can see - so I don't understand what is holding them back.
Valid points - maybe they need advise on how to allocate budgets, maybe they underestimated what is required and thought witnesses are enough. Sure they learn from that experience. Heard similar thoughts by people i respect and appreciate your thoughts. Maybe a young company needs to listen to old tech farts too. Re money, no idea what was the issue - maybe the plan a present for all Steemians and hence saved it for that 😎
I think it partially comes down to where people were trained. It is entirely possible to learn computer programming without learning systems engineering and architecture - plus also without learning structured design principles of the type used usually in high value/risk projects.
I don't know what methodologies are being used here, but the absence of diagrams and documentation is a clue to me that there isn't likely to be much available.
Agreed - ib my job I am a fan of project management reporting tools - better to coordinate team work. Voting btw after rc / sp loaded again - not that u think i would not appreciate - usually voting all useful comments 🌍
hehe - no problem, maybe things will be normalised in another 3/4 days - we shall see.
i feel it's important to have actual system architects involved, alongside project managers - to ensure the vision is clear and maintained. i know though, that from a developer's perspective, the top-down approach can feel unpleasant too. balance needs to be found.
I am glad I still follow you.
Your diagram made me laugh and I may use it in other areas.
When the dust settles more they may be open to some suggestions.
I am just glad we are getting through this. I do view Steemit from a different perspective now.
thanks for the laugh I needed it in here....
I chose you for one of my few comments for today. 🙏
Oh, thankyou. I do my best. :)
Great explanation about the things that needs to be improved. I hope that because of this terrible Hard Fork everybody is more aware about the issue that witnesses and Steemit.inc developers needs to work better and more efficiently together. Transparency and one location to go website/chat-room where alle Steem developers can discuss the code and work on it (not only Github) would probably be beneficial for everybody. Good informations thanks for sharing @ura-soul
You are welcome - let's see what transpires. Community pressure needs to mount too.
You got my first upvote since the HF.
It has been frustrating not being able to even vote for 2 days and I have over 3k SP.
Oh, thanks a lot. Have one back!
That flowchart diagram is pure gold :)
hehe - I have an archive of rare classics.. like some kind of unknown master meme DJ :)
Creo en Steemit y pienso que todo lo que están haciendo es para mejorar, veremos sin duda grandes avances. Adelante!