New features: extended phishing protection, preview and scan of shortened links

2 new features added to @keys-defender:

Logs

New feature # 1. Shortened URL preview & scan
This defense mechanism helps to prevent users from being tricked into navigating to a malicious site (eg. phishing, malware).

Example: _{(on Steem - my testing playground)}

So, as per screenshot above, the unfurled URL gets also checked against the known phishing domains as if the link in the comment/post was not shortened to begin with.

At the moment, the preview is automatically generated for the following URL Shorteners:

- https://tinyurl.com/
- https://goo.gl/
- https://bit.ly/
- https://ow.ly/
- https://buff.ly/
- https://is.gd/
- https://adf.ly/
- https://bit.do/
- https://rb.gy/
- https://rebrand.ly/
- https://polr.me/
- https://b.link/
- https://t2m.io/
- https://gestyy.com/
- https://zpr.io/

More will be added in the future.

Furthermore, comments and posts that include links using http instead of https will now get a warning too.
To prevent spamming, this comment gets out only 1 out of 10 times since apparently many users share HTTP (no HTTPS) links.

Example: _{(on Steem - my testing playground)}

NOTE: feedback is welcome! Please let me know if it gets too spammy and I'll reduce these type of comments even further!
_{(As per screenshots, users can already prevent future comments replying OFF if they're bothered by it)}

This is also the first step into not providing my services for steem users: steem accounts that do NOT have the same keys on Hive won't be notified by this bot when they come across a link that is flagged as phishing in my database.

So.. sorry, this feature is only for Hive users. Steem users, you already have Justin Sun taking care of you and making sure you are safu! (lol)

See also my last post on @gaottantacinque for more details and a code snippet you can re-use in your project for the same purpose.

                   

UPDATE 1:

• Added whitelist of shortened URLs. Eg. Actifit uses 2 bit.ly links in all their automated posts.
• While the shortened urls preview and scan feature was working well on steem, it seems to have some issues here on hive as some of the requests required in its steps fail due to node issues. Will investigate further and work around those with a temporary patch. PS. Decided for the time being to remove the checks to prevent multiple replies when the author edits a post/comment containing a shortened URL. Will bring it back when the nodes issues are gone.
  

UPDATE 2

• The auto-replies with the shortened URLs previews got a couple of OFF as reply so I decided that until I preview all shortened links in the post in a single message and keep in the cache users that I recently notified, I will only preview shortened links in comments and not in posts. This should reduce the clutter by one order of magnitude.
• Auto-replies for HTTP links reduced from 1/10 to 1/20 to further reduce spam.

UPDATE 3

• HTTP notifier updated to check whether automatic redirection to https is in place for the http links it finds. If it is, the user does not get notified as it's already safe. In all the other cases the user gets notified: the http link does not work, its https site exists but the https redirection is not in place, its https site exists but it errors out.
• Now periodically (every hour) retrieving and checking against @guiltyparties's list of phishing/compromised domains..

..and phishing users..

..in order to counteract these threats!!

_{Example of discord notification for known phished users' activity}

---

_{Keys protection main features:}
- _{Introductory post for @keys-defender launch}
- <sub>Automatic-posts on leak detection, weekly reports

Other features:</sub>
- _{Phishing protection}
- _{Re-posting detection}
- _{Shortened URLs preview and scan}

_{My past security disclosures (from most recent):}
- _{XSS vulnerabilities in hive-db.com}
- _{XSS vulnerabilities in scribe.hivekings.com}
- _{XSS vulnerabilities in hiveblockexplorer.com}
- _{Malicious ads redirecting all Steemit iOS users to a phishing site}
- _{Reverse tabnabbing and clickjacking in steem.chat and steeemit registration page}

 

---

To support this bot..

                                       

- _{Delegation links:
10, 20, 30, 40 HP
50,100, 200 HP,
500 HP, 1000 HP}

- _{Curation trail
Follow my curation trail on hive.vote to upvote all my posts with a fixed weight.}