Steem Data Services (SDS) / Upcoming Update Info / Version 0.1.7a

in #sds2 years ago (edited)


Version Details


SDS will be updated to version 0.1.7a within the next few days. Currently re-indexing of all blocks is already running at the sds0 endpoint (developer requests going to sds0 are automatically proxied to sds1 in the meantime).

Here are some of the most important new features and changes in this version:

  • Field word_count in all feed lists
    This will allow us to get the word count for each post directly from the result without the need of rendering the Markdown and stripping Markup tags from the body. It is not perfect yet, but will be optimized over the coming weeks.

  • Feeds by hot/trending score without whale votes
    The newly added trending score algorithm allows to get hot/trending posts without whale voters. It will enable us to retrieve lists of trending posts with almost all 'up-boted' posts removed.

  • Feeds by interaction score
    The new interaction score algorithm allows to get trending posts by user interaction. Instead at the voted rshares (payout) of each post, it only looks at author and reader interaction (more details will follow in an extra post at some point).
     
    The algorithm will be optimized over time (@realrobinhood will contain a small overview in his coming daily stats reports, where the results can be discussed to further improve the whole thing).

  • Field image_url deprecated
    The field image_url in the feeds_api's results will be removed (returned empty for now and removed completely in one of the next updates), because it can't be reliably filled on the server side. For example, getting preview images for videos should always be done by the client, because it depends on the used device's screen size which resolution should be loaded (mobile devices should load a smaller preview).
     
    Some video platforms (like vimeo) don't even allow loading preview images without making an extra request to their API. Additionally, not all front-ends will fill the image field in json_metadata correctly, so it does not make sense to have the logic implemented on two ends. The viewer app needs to be able to get the image from the body anyway. Therefore I will remove this redundant field from the API.

  • Improved getSteemProps API method
    As many developers requested to be able to retrieve a bundle of most important Steem properties with one request, I added the following API method (ready to be used on sds0, sds1 will be updated in a day or so): https://sds0.steemworld.org/steem_requests_api/getSteemProps
     
    The result also contains a field steem_per_share that provides the current STEEM per VEST value. Currently the data is updated every minute, but I will change this to be almost in sync with the blockchain. I also added a few more fields like the current market ticker data. If you should miss any property in there, just let me know ;)

Status


  • [x] Re-index all blocks at sds0 endpoint
  • [x] Copy parsed data to sds1 endpoint
  • [x] Copy parsed data to main sds endpoint
  • [x] Replace second steemd_ah instance with new lite (500 GB) version
  • [x] Make use of the new getCommunitiesBySubscriber method and show the user's subscribed communities in the Dashboard on SteemWorld

I wish you all a super inspiring week! Steem on ~ 🚀


If you like what I do and you want me to be your Steem witness,
please vote for @steemchiller on steemit.com/~witnesses or steemworld.org/witnesses.


Sort:  

thank you friend, this new update is amazing in my opinion this is very good for the future on the steemit platform in the future. Thank you very much for your hard work sir and thank you also for providing a platform to make this program a success.

and I try and also hope that all of you need your help in the extraordinary hard work so far and I also want to know more in this steemit one of which is how I get good steem to do power ups and we need it I hope always happy to meet you you sir.

good thing steemworld has helped us a lot to make things easier.

  • thank you very much sir @steemchiller hope you are always in our hearts. 🥰

I am always excited to see your update Sir @steemchiller and trust me, this is one of the best. This new update is so great and interface-friendly. Thank you for all your great Jobs, Sir, and thank you for contributing to the success of this platform.

I hope to offer any little assistance wherever is needed someday sir.Thanks for your hard work.

Muchas gracias por la información, excelente trabajo actualizado por la calidad. Saludos!

Hello @steemchiller this is the account that my daughter used, I had to borrow it to write to you because due to a silly mistake on my part I accidentally clicked on a link, I thought my account could be hacked and I wanted to change the password, but in the process I didn't copy the new generated password and I accidentally hit accept. The key was changed and I had not copied that key. Is there any way to recover my account?

Could you please help me, I beg you, I want to recover my account.

Last night one of the guys who is with me from the curation gave me your Discord contact and I sent you the friend request, if you want we can communicate by that means. Thank you

@daytona475

Hi, please start the account recovery for @daytona475 on https://steemitwallet.com/recover_account_step_1 with the previous password (the one you still have) and the email address that was used when signing up the account on steemit.com .

You have 30 days (from the moment you changed the password) until the account can no longer be recovered. In case you have any questions later on, just let me know.

I did what you told me, yes, I have the mail. I expect a prompt response to recover my account... Thank you very much for your help! God bless you!

I placed the email with which I registered and I still haven't received the email, is it normal for it to take so long?

Normally, the email will be sent out automatically (only when the email address matches the one from the registration process). Let's wait a bit and see if it will arrive later today. In case it should not arrive, I will contact the Steemit team to start the recovery manually.

Wait until now to check my mail and it still hasn't arrived, I even checked my spam folder and nothing. Before recovering my account I made sure to put the correct email, I checked and I have all the emails that have come to me from Steemit since I registered. So I have no doubt that I put the registration email.

The following image was the email I received when I registered

image.png

Thank you!!

Hello, I still haven't received the email to recover the account :(

I reported your case to the team, so you should receive the email within 48 hours. You don't need to worry. As long as you have the old account password, we can recovery your account 100%.

Thank you very much for your valuable help, I finally have my account again. I am very grateful!

God bless you, have a happy day!!

Great work. Many thanks to your effort.

yes Mr @steemchiller always release information that is very important for us to know the Steem Data Service. thank you very much for providing more information regarding Steem Data Services (SDS) / Update Notice / Version 0.1.7b .

Thanks a lot for the update information, Mr. @steemchiller
Great work!
Steem World has helped us many things.
We support you 100%
🥳🥳🥳

Hi @steemchiller,

I have been working on the deep learning curation bot, and did not like the first version I created (it was alright at predicting article values, but not great). So I've gone back to add more complex features, and this time I'm using historic data on the author and the top/median curators. One problem I've run into is that it's really slow due to rpc errors. I've realized a way I could avoid the rpc errors is by setting up my own private node, but I don't know how much storage that takes, and wanted to ask you since you're a witness and run your own node. Do you have any advice on setting up a node if it is possible for my computer (if not perhaps I could invest in cloud storage if it's not extremely expensive)?

Thanks for your help!
@cmp2020

Hi, the requirements for running an own node depend on the APIs you want to use. In general, you won't get far with a 500 GB disk anymore and for most common cases I would recommend at least a disk with 1 TB SSD space. The amount of available RAM is not so important (32 GB would be good, but even 16 GB would currently work without any issues).

If you also plan to additionally run something like a Hivemind node (in case you are making use of the Bridge API methods), which receives the blocks from a regular full node, you will need even more disk space, of course.

So, the first thing to do would be to find out which APIs your current (and maybe coming) projects will require to be realized. Basically, it's always more fun to work with a fast SSD (for best experience even NVMe). In case you should decide to run an own node, I recommend to use WebSocket connections instead of normal HTTP requests, because this will perform way better when running many requests in a row ;)

Here are some links that might help you to get started faster:
https://files.steem.fans
https://github.com/steemfans/steem-docker-ex

Thanks for your response! Unfortunately, my computer has a maximum of 16 GB of RAM so I think I will have to figure out a different way to do this until I have a proof of concept to show that it would be worthwhile to invest in either cloud computing or a computer with more ram. I did invest in 2 TB of external storage today though. I would appreciate any recommendations on how to download data faster with what I have. I am considering just cutting the historic data for now and working with the other features I've added.

Thanks again!

Edit

We decided to try setting up a node on our extra computer (which we hardly use) with 16gb ram and the external storage I bought. So far we haven't run into any problems so we might succeed at setting up the node. My dad and I just didn't think it would work because we're running it on WSL, but so far it seems to be working.

Yet another edit

We got it working, and have started syncing the blockchain. Thanks for your help! I hope to post an update on the ai soon enough ;)

Hey greetings @steemchiller gorgeous guy i have a problem so that i am discussing with kindly help me .
Dear whenever i tag any user than he didn't get my tagged notification for example i am tagging a my friend @shafqathussain but sadly to say he didn't get my notification what is the cause kindly i shall be very thankful to you look at it
Screenshot_2022_0327_150521.png

Hi, that seems to be related to steemit.com . When I switch to @shafqathussain on SteemWorld (https://steemworld.org/@shafqathussain), I see this in the Mentions section:

Screenshot at 2022-03-27 18-47-54.png

thank you for give info that thank you sir have a nice day Greetings...😃

This is an issue on steemit. There is a limit on the total number of people who get notified when you mention them in a post/comment. If you have mentioned more than 10 users in a single post/comment, only the first 10 will receive the notification. All the others are ignored unfortunately.

Yes it is, but i mentioned only user and no notification receive to him

Danke für die ganze Arbeit!
Habe mich immer gefragt, wie Leute hier einen %-Vote verteilen, sehe gerade das man per Manual-Vote das ganze auf SteemWorld realisieren kann!
Werde ich auf jeden Fall nutzen mit ansteigender Voting-Power! :)

Your post is manually rewarded by the
World of Xpilar Community Curation Trail

STEEM AUTO OPERATED AND MAINTAINED BY XPILAR TEAM
https://steemit.com/~witnesses vote xpilar.witness

Coin Marketplace

STEEM 0.27
TRX 0.11
JST 0.031
BTC 68396.79
ETH 3797.89
USDT 1.00
SBD 3.56