>>37It would index only what I explicitly tell it to and wouldn't be shared. In fact, that's why I'm apprehensive about making it. Most sites that have a lot of information to search for already have a search feature designed for that site. Search engines are useful for indexing large numbers of lesser known sites. That doesn't fit into my design principle since I only use a handful of sites anyway. And things like blogs can be followed with RSS. This makes me question whether I need a search engine in the first place. The whole purpose would be to index the few sites I use that don't have a search feature.
/prog/rider is one example since it doesn't have explicit searching and it's easy to lose track of memorable posts. But making a whole search engine just for those handful of sites probably isn't a good idea becuase it's trivial to write a basic script to search them. I can write a one-liner in a few seconds to find anything on here.
If I'm programming, it's a bad sign when I have to look up anything that isn't documentation. If I can't find the answer on Stackoverflow and its sister sites and have to resort to looking randomly through the web, that's desperation. We are at a point in time where the Internet is centralized and many sites are broken and malicious. They require Javascript for basic functionality and employ tools for spying on users. It's not wise to visit random sites because most of them won't be useful and will require the user to give unreasonable permissions to a site just for the sake of reading plain text.
Why do I need to enable Javascript and connect to Google Ajax, Google Fonts, and Google Analytics just to find out what some error means? Why do I need Javascript, cookies, and cross-site requests just to view a bit of text? Why do I need Javascript just to click on a link? Why do I have to spend so much time messing around with my Firefox addons when I only want to take a glace at a site that I'll probably never visit again? I don't need animations and fancy fonts. It breaks the whole site is ridiculously difficult to parse which makes things like using web crawlers much harder. If a site is so hard to use, there's probably nothing useful there anyway.
Search engines aren't so much for searching as they are for reducing the number of clicks it takes to get somewhere. Or even to use a widget to check the weather or some basic math. If I had a search engine, I would just use it as a replacement for bookmarks. Most people know where they are going, they just don't feel like typing. They don't type in "example.com", they just type in the site name and click on the first result. Of course, the first result is often a paid ad for some kind of malware, which explains why it is still so prevalent. If you use a !bang in Duckduckgo, it will redirect you to a search in another site with the query appended into the url. That's simple and useful.
Imageboards are transient and not worth searching. There's usually a JSON API and a catalog that can search thread titles. There are also archives which have a search feature. Textboards can be searched with a script.
Every wiki has a search feature. So do social media sites and online stores.
Most software can be found on code hosting sites like Github. Otherwise, there is typically an obvious homepage.
Every good programming language has its own site for searching documentation. Stackoverflow is where errors are pasted.
All the boorus have searching and tagging built in. They are search engines, except explicitly for anime pictures. There's services like Tineye for doing reverse image searching.
So making a search engine just wouldn't be that useful. It would be a better use of time to make alternatives for sites that are crudly designed. That was my inspiration for wanting to make a search engine in the first place. I wanted to see one without ads, spying, or censorship and one that demotes websites that have Google services, Captchas, and social media widgets.