My first time using Terraform! Petmas creates a curated Twitter feed of Christmas-y pets.
The elves went on vacation, it’s just me @gracenng
2 Likes
Infrastructure as code feels pretty magical. Overall the project kept me busy during the holidays and I’ve learn so much more than I could imagine. 
In the future, I hope to resolve the Twitter CSS loading issue, as well, an autoscalabling Kubernetes infra that allows the users to trigger the scrape would be nice.
Apologies I said “we” in the video, just me in the team, a habit from hackathons.
A note of gratefulness to Illya T. for design the cutest cat logo, and my Waterloo ECE friends for their mentorship.
Link: > React App
Github: > GitHub - gracenng/petmas
Video demo: > https://youtu.be/vzbWrAY2muI
Connect with me on LinkedIn: > https://www.linkedin.com/in/grace-n-nguyen/
4 Likes
Hey Grace,
Nice project, I have a few questions and remarks :
-
what do you use Selenium for?
-
in the HTML you insert, you add multiple times this resource :
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>, this is unnecessary as you ultimately only need it once. I suggest you to move it in your template directly. -
injecting html in a webpage that way could lead to a security issue (you don’t control the content!). I recommend you to build your own embedded code.
If you take a look at how the Twitter embedding script is working, you end up seeing that they targetblockquote.twitter-tweetelements in the page.
That means if you write this :<blockquote class="twitter-tweet"> <a href="https://twitter.com/TheTzila/status/1342581492149469184">Example</a></blockquote>it will render your tweet properly.
I made a quick example here so you can see it working.
Then when you fetch the tweets, you just need to save in your db the content of the tweet, the user and the link to it (in a JSON format for instance) and then with your React app you can build the above HTML from scratch (that way there is no more HTML content injection in your website).
Hope that helps, let me know if you have questions 
Wow, Grace, this is like the coolest thing, Twitter is so much better when it is full of pets rather than the usual stuff.
1 Like
Hi,
Thank you for your questions. I used Selenium to drive the scraping process on publish.twitter.com which gives me the html blocks in return. In retrospect, I could have also just use Twitter’s Oembed API and get the same result. Thank you for the example! I didn’t know that was the format so I just stuck with what Twitter gave me. I tested the tag in React and it doesn’t make a difference but it is redundant and your way definitely make it so much more simple.
Thanks for the input!
Great! Let me know when you have everything working, I’d like to see the result, happy coding!
1 Like