Setting up this website, the troubles, the fun and the why
Hello! This is my brand new website! If anyone remembers my old one it was okay, but not great. I had a blog and project system which worked but I ended up having some issues with the database and all the data was lost. I have been planning to rewrite everything from scratch for over a year now. And I finally have! I decided to properly learn how to use the tools at my disposal to make the best website I can manage. Parts of this were a pain to mess with, but it was a great learning experience and overall was fun.
Why?
As I mentioned, the old website was not great, the frontend was done by me from scratch while learning how to use NuxtJS, which I did enjoy, but it was a little too much for my needs and I suck at frontend development. I love doing backend stuff a lot more, where I do not need to worry about any UI or design. The old backend and the account system was not great either, I had originally spent a long time trying to get it working properly. I got it working but it was not that good.
Making the website
Backend
I started with the backend as that is what I enjoy doing. I have been learning how to properly write and host a Springboot backend with my other project MineCrash under Eimer Archive a couple months ago which really helped. I wanted a very basic backend that just had blog and account support as a base so I do not spread out my focus too much and actually finish something.
Database
For the database I went with MariaDB and Liquibase to handle managing database schema changes. That is one of the things I would get stuck on and overthink, “how do I manage database changes?”. By using Liquibase it was all handled for me easily and I did not have to worry about it.
Testing
Automatic tests is something I have never touched before the backend for this website too. The tests are used to make sure everything is running properly after you have made changes. They seem very useful but I had just never gotten around to learning how to use them. I decided it would be a good idea to write some so I could spend less time manually testing and more time writing new code and features.
It was actually pretty simple to use. You just make a little method for each test, make sure mock instances of some classes are set up and you are ready to go! You can make requests to and endpoint and check if what is returned is what was meant to be returned. Or you can directly check what a method returns which is usually lighter and faster to run. I played around with it for a little bit and it was nice, automatic testing so I do not accidentally push any broken changes to production. The only problem is it is a bit annoying to manually go in and make a test for each case, but once it is set up you shouldn’t need to touch it often.
Accounts
Accounts are important, they are needed if you want to be able to interact and have an effect on a website, like leaving comments. Or can be useful for just admins to manage stuff. Security is a big part of this too. For this I learnt how to use JWT so it could easily handle account tokens to people. It was interesting, with my previous set up I was generating a string that would act as the token and storing it in the database along with the account it was for and an expiry date. This one is fully client side, it is stored on the client and is nowhere in a server side database. The server only take a master key that can encrypt and decrypt these tokens, the tokens themselves contain all the information needed, like the account it is for and the expiry time. The server then validates it whenever it is sent along with a request. This was pretty interesting to me.
A problem I had while working on accounts was CORS. I hate CORS. It is made to bring pain into this world. It is something about restricting access of websites to themselves and not other websites I think. This is what I spent FOREVER trying to get to work with my last account system, so I was not looking forward to messing with it this time. I tested my signin/signup endpoints using Postman and it was working fine. At this point I had a basic frontend to test with and that is when CORS started laughing evilly in the corner. It was refusing my connection. I do not remember much about what I actually did, but I managed to get it working after a couple hours of torture. My past experience certainly helped but it was still not fun. That was the end of my CORS problems, for now…
With that, the current version of the backend was basically complete and ready to be hosted with some minor tweaks.
Frontend
The frontend is where I am pretty bad. I am horrible at designing stuff. This website was mostly a template (At this time anyway) that I have modified to be my own. To be fair if you call this ripping off a template than anything with basic shapes is also a ripoff.
For my last frontend I used NuxtJS. I do not remember why I chose it but I enjoyed it mostly. I had not touched much frontend development or NuxtJS in about a year, I have heard good things about Svelte so I decided to have a look at that. I began working on a project in it but then someone mentioned Astro. It said that it was compatible with many other frameworks if you needed some of their functionality so I decided to try that instead. So far I have enjoyed it, it is pretty simple to understand and easy to use. I would recommend checking it out. I needed some help understanding how to do something and their discord responded quickly which was great.
The frontend is just the blog template but modified with new pages and functionality. The design is the main thing I took from it. How blogs work has been completely changed so that it works with my backend. Using Astro has actually been pretty enjoyable as I have been learning it.
Hosting
Once I had the backend and frontend all setup and seemingly ready to go, I went to my Hetzner VPS and started them both up after some minor configuration for production. I set up an nginx reverse proxy for the frontend and backend so it worked through my domain, set up Let’s Encrypt for a secure connection and went to the website!
The first problem
I refreshed a few times to test the speed and other pages but suddenly, sometimes the page refused to load. It would go for a whole minute then fail. It would often work if you refreshed it once that occurred but this was a major problem, it made the website barely usable. I had this problem both on the frontend and backend, the VPS resource usage was very low as well, with less than 5% on the CPU. The possible culprits were Cloudflare who was handling the domain, or nginx who was handling the reverse proxy. I initially though it was some issue with Cloudflare but all of my other websites and domains were fine, only ones hosted on this specific VPS were having issues. It took a while but I managed to narrow it down to the server. I was testing some different configurations for it out but nothing was helping. I checked the nginx access and error logs but did not notice anything wrong at first. I was complaining about it in the LibGDX Discord server, kindly James was helping me out a bit, testing, giving feedback on some other issues he noticed. When he mentioned this
He mentioned looking at the nginx access/error logs. Which I already had, but this made me focus on it a little more. I refreshed my website until I got the long loading issue and was watching the access logs, nothing happened. I switched to the error logs, got the long loading time and waiting for it to timeout at a minute. This is where an error was spat out.
2024/11/08 11:14:31 [error] 14576#14576: *1767 upstream timed out (110: Connection timed out) while connecting to upstream, client: X.X.X.X, server: api.imjustdoom.com, request: "GET /api/blogs HTTP/2.0", upstream: "http://[::1]:8085/api/blogs", host: "api.imjustdoom.com"
I was looking at it and noticed that the “upstream” url looked weird. It had [::1] instead of localhost or 127.0.0.1 which is the address for the local machine where the websites were running. Turns out this was an IPV6 address instead of an IPV4 address like 127.0.0.1 is. localhost would randomly switch between converting localhost to an IPV4 and IPV6 request, and when it became an IPV6 request it would make the request time out. This is because I disabled IPV6 requests on my VPS when I set it up as I did not think I would need to do anything with IPV6 any time soon… Basically IPV4 and IPV6 are different ways an address can be specified. IPV4 has less possible combinations and is running/has run out (You can learn more here. It has about 4,294,967,296 addresses with many being reserved. And IPV6 can have about 340,282,366,920,938,463,463,374,607,431,768,211,456 address, just a little more than IPV4.
So yeah, basically this was me shooting myself in the foot, forgetting about it, and wondering why my foot hurt. I ended up just changing localhost to 127.0.0.1 in my reverse proxy configs to solve it. I spent maybe 2-3 hours trying to fix this that day. It was not fun. You know that feeling of happiness and success when you figure out an annoying bug in code? Well what I had was like that, but instead it was sadness and disappointment. I was glad it was over but I was also so sad that I spent that long trying to solve it. I had actually encountered this issue with the backend of my MineCrash bot when making requests through the domain, but I left it alone because the Discord bot made local requests and I had not set up the endpoints properly for public use yet. So this was an issue that has been annoying for months. The good news is that the website feels pretty nice to use now, and I can continue improving instead of trying to solve this annoying issue.
The CORS problem
While I was having trouble getting the website to load, before it was solved. I also found out I was having CORS issues only in production on top of that. You thought we were done with CORS? You are never done with CORS… It will always come back. This time it did not take too long to fix though, I was looking through some people who have had the same problem and noticed I had not added cors(Customizer.withDefaults()) to my security config. I had written my CORS config but I had not properly enabled it. With that it was solved. It took maybe 20-30 minutes as I was dealing with the previous problem as well making it annoying to get a request through for testing.
Ending
Thank you for reading until the end, or even clicking on this page and skipping to here! I am bad at writing so this may not had read well but I hope you enjoyed it. I will try to write more blogs in the future and continue improving my skills. Don’t forget to join my discord server! You can provide feedback on the website, suggest blog ideas or just stop by and have a chat!
Short term goals
Here are some features I am wanting to work on for this website over the next few weeks:
- Account settings - Includes password reset, changing email
- Dark mode
- Projects list - Projects will have a page with the description and update information, maybe even downloads too
- Blog comments
or reactions In website blog poster/editor/preview