update for beem: handling Could not find API exceptions and other node failures

in beem •  4 months ago 

Repository

https://github.com/holgern/beem


beem-logo

beem is a python library for steem. The current version is 0.20.23.

There is also a discord channel for beem: https://discord.gg/4HM592V

The newest beem version can be installed by:

pip install -U beem

Changelog

  • Switch to next node, when current node has the necesary api not enabled
  • handle Client returned invalid format. Expected JSON! and switch to next node
  • More checks added
  • get_estimated_block_num is faster and uses BlockHeader
  • exclude_limited=False is default now for get_nodes

Switching automatically to the next node on ApiNotSupported errors

At the moment, the full nodes have some problems and a lot nodes have disabled some apis. beem can now handle this and is automatically switching to the next node in the list.

from beem.account import Account
from beem import Steem
stm = Steem("https://steemd.minnowsupportproject.org")
account = Account("steemit", steem_instance=stm)
print(account.get_blog(0, 1))

returns the error:

beemapi.exceptions.ApiNotSupported: Assert Exception:api_itr != _registered_apis.end(): Could not find API follow_api

As there is only one node given, beem cannot switch to then next node.
When using a node list instead:

from beem.account import Account
from beem import Steem
from beem.nodelist import NodeList
nodelist = NodeList()
stm = Steem(node=nodelist .get_nodes())
account = Account("steemit", steem_instance=stm)
print(account.get_blog(0, 1))

the script returns:

Error: Assert Exception:api_itr != _registered_apis.end(): Could not find API follow_api
Lost connection or internal error on node: https://steemd.minnowsupportproject.org (1/100) 

[<Comment @steemit/firstpost>]

which means that first failing with https://steemd.minnowsupportproject.org, but then retrying sucessfully with https://rpc.usesteem.com.

Using update_nodes

The following nodes are currently stored in NodeList:

from beem.nodelist import NodeList
nodelist = NodeList()
print(nodelist .get_nodes())

the following nodes are returned:

['https://steemd.minnowsupportproject.org', 'https://rpc.usesteem.com', 'https://anyx.io', 'https://api.steemit.com', 'https://steemd.privex.io', 'http://anyx.io', 'https://appbasetest.timcliff.com', 'https://api.steem.house']
from beem.nodelist import NodeList
nodelist = NodeList()
nodelist.update_nodes()
print(nodelist .get_nodes())

nodelist.update_nodes() fetches the currently working nodelist from @fullnodeupdate. I'm writing a fullnode benchmark every 3 hours to this account, which can be obtained from the json_metadata field.

Currently the following nodes are working:

['http://anyx.io', 'https://anyx.io', 'https://steemd.minnowsupportproject.org', 'https://api.steemit.com', 'https://rpc.usesteem.com', 'https://api.steem.house', 'https://steemd.privex.io']

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

handle Client returned invalid format. Expected JSON! and switch to next node

cool! I was also getting this error many times these days. What's the exact logic? a block is always fetched as a whole block? So no duplicated transactions are possible?

When a node returns an invalid json, I'm switching the next node in list and retry the some call, so no duplicated transaction possible.

Thanks!

Big fan of the beem library. Keep up the good work, brotha.

Thank you!

I finally started to learn Python while debuging your scripts and writing my own for STEEM automation. 😂

For those (like me) who forget how to upgrade pip packages.

pip3 install beem --upgrade

Successfully installed beem-0.20.23

Thank you so much for participating in the Partiko Delegation Plan Round 1! We really appreciate your support! As part of the delegation benefits, we just gave you a 3.00% upvote! Together, let’s change the world!

This post has been just added as new item to timeline of beem on Steem Projects.

If you want to be notified about new updates from this project, register on Steem Projects and add beem to your favorite projects.

Hi, @holger80!

You just got a 4.3% upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in here to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.

Hi @holger80!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 7.610 which ranks you at #48 across all Steem accounts.
Your rank has improved 3 places in the last three days (old rank 51).

In our last Algorithmic Curation Round, consisting of 366 contributions, your post is ranked at #141.

Evaluation of your UA score:
  • Your follower network is great!
  • The readers appreciate your great work!
  • Try to work on user engagement: the more people that interact with you via the comments, the higher your UA score!

Feel free to join our @steem-ua Discord server