Commit graph

143 commits

Author SHA1 Message Date
Yorhel
50fe17a604 Indexer: Support .deb archives 2016-11-15 21:15:35 +01:00
Yorhel
1f05463c3a About page: Remove TOC feature as planned 2016-11-09 19:01:24 +01:00
Yorhel
aa01365e60 Move nav menu a bit up to create space
This is where the old nav menu used to be. This involved shrinking the
width of the locations/versions selector, but that never needed the full
page width anyway. Unfortunately I suck at CSS so the nav menu and
selector thing won't look too great on smaller screen sizes; but that's
just a minor visual uglyness.
2016-11-09 18:58:34 +01:00
Yorhel
09af881767 Add TOC listing + more section/lang select back into a nav menu 2016-11-09 18:43:10 +01:00
Yorhel
20141aa980 indexer: Improve charset detection + lower file cache time 2016-11-09 18:41:53 +01:00
Yorhel
7d2abfb3a4 indexer: Fix storing locale as NULL when empty
Perhaps it's better to get rid of NULL and make empty the default value.
But for now this'll do.
2016-11-06 16:24:45 +01:00
Yorhel
cb81bedac1 Add arch/encoding metadata to DB + Fetch Arch Linux x86_64
The encoding metadata will be very useful in finding badly decoded man
pages. The package 'arch' is necessary to properly identify which
package was used, which is not obvious now that I'm going to switch more
systems to the (more common) x86_64 arch.
2016-11-06 16:05:16 +01:00
Yorhel
b8a1945d38 Merge branch 'indexer' 2016-11-06 15:26:42 +01:00
Yorhel
5e39af459f Replace old Arch Linux scripts with new indexer 2016-11-06 15:26:20 +01:00
Yorhel
1ca43665a1 indexer: Add file caching + Arch Linux indexing 2016-11-06 13:34:22 +01:00
Yorhel
35fab522d6 Indexer: Support HTTP fetching + misc improvements 2016-11-06 09:21:53 +01:00
Yorhel
aff68205b0 Add postgres package indexing + cli options 2016-11-05 10:22:31 +01:00
Yorhel
0cab758665 Add support for man page reading & decoding 2016-10-30 11:06:14 +01:00
Yorhel
c8bb4da246 Use libarchive3-sys crate directly + improve archread API
This all should offer a more convenient and robust interface to handle
all sorts of archives.
2016-10-29 09:33:39 +02:00
Yorhel
9db73b2709 Fix CSS I accidentally removed 2016-10-26 21:31:47 +02:00
Yorhel
863fae2476 Add link to manpag.es 2016-10-26 19:27:57 +02:00
Yorhel
25a39c6fe4 Improved pagination on package info pages 2016-10-26 19:25:23 +02:00
Yorhel
022e9acc4f WIP: Rewritten man page indexer in Rust
Currently just figuring out how to read archives. Turns out to not be as
simple as I had expected.
2016-10-22 14:54:37 +02:00
Yorhel
965aa9a2f6 Add Ubuntu 16.10 2016-10-19 07:30:49 +02:00
Yorhel
7535218a06 Add FreeBSD 11.0 2016-10-18 07:09:27 +02:00
Yorhel
a7352d27b9 Fix possible wrapping of MANNEDINCLUDE by removing space
This doesn't really guarantee that it won't wrap, but fixes at least one
man page.

- https://manned.org/BlockSelectionDCOPInterface/6dfdf921
2016-10-16 10:28:44 +02:00
Yorhel
5436435c3f Improve handling of man names with special characters
The 'source' link was broken for mans with [ or ] characters.
All links were broken for mans with space characters.

Man page of the week:
https://manned.org/KGenericFactory_%20KTypeList_%20Product,%20ProductListTail%20_,%20KTypeList_%20ParentType,%20ParentTypeListTail%20_%20_/dfc33ca6

There's a 5 man pages left with a '%' or '#' character. I've no idea if
it's worth handling those; A fix for these isn't going to be as trivial
as this commit.
2016-10-16 10:19:27 +02:00
Yorhel
8a0fac08b6 DB cleanup: Remove some non-manpages & fix wrongly-detected locales 2016-10-16 10:03:34 +02:00
Yorhel
17fc298217 Fix handling of URLs ending in a ⟩
I've known about this issue before, but didn't realize it was so
widespread. This fixes many links.
2016-10-16 09:11:15 +02:00
Yorhel
7d31f41ba8 Add FreeBSD 10.3 2016-10-15 22:37:58 +02:00
Yorhel
4affcec7c3 Homepage: Add "less" button after clicking on "more" 2016-10-15 16:55:23 +02:00
Yorhel
6740dc2546 A few more links to other man page sites 2016-10-15 16:46:03 +02:00
Yorhel
44df29ea18 Fix 404 on /(pkg|man)/<hash> 2016-10-15 16:06:18 +02:00
Yorhel
20daba820f Complete revamp of navigation menu on man pages
This removes the navigation menu on the right, leaving more space for
the actual contents. Instead, there are now a few links/tabs at the top
of the page. There's also a 'permalink' now.

The previous navigation combined the selection of man page versions,
translations and sections in a single menu. While handy in some cases,
in most cases it was just slow and messy. It also didn't scale very
well, some man pages have so many versions that it significantly
affected the page load time.

The 'locations' table has now also been moved into tab and is loaded
asynchronously as well, for the same performance reasons.

I had hoped that this new navigation would be much easier and more
convenient, but honestly, it's still a mess. At least the new code is
more maintainable, so perhaps I'll be able to make some incremental
improvements in the future.
2016-10-15 16:06:18 +02:00
Yorhel
3f40896679 Add FreeBSD 10.2 2016-10-14 08:09:53 +02:00
Yorhel
c04e6b3b6a Add FreeBSD 10.1 2016-10-12 17:02:37 +02:00
Yorhel
1106b0c08d Add FreeBSD 10.0 2016-10-10 17:19:08 +02:00
Yorhel
b7328cc039 Reorganize links on homepage a bit 2016-10-09 11:34:55 +02:00
Yorhel
8817f5c252 Fix typo on info page 2016-10-09 09:41:35 +02:00
Yorhel
b023bec0cd Use placeholders in search SQL
I didn't do that before in order to ensure that the planner could always
optimize the LIKE queries, but it's optimizing them just fine with
placeholders now.
2016-10-09 09:33:45 +02:00
Yorhel
3eb1cbae63 Log slow pages 2016-10-09 09:26:01 +02:00
Yorhel
fcaccd6f48 Document some URLs + add URLs to link to specific man pages 2016-10-09 09:23:16 +02:00
Yorhel
659b7afece Implement man selection algorithm in SQL + fix various related bugs
Man selection has to be performed over several thousand rows in some
cases. Loading all those in Perl and then doing the selection isn't very
efficient[1]. The getman() implementation was also buggy: The comparison
function used to determine which man page should be preferred was not
associative[2], and the result thus depended on the order in which the
man pages were compared. This resulted in some wrong selections in some
cases.

While I was at it, I also made the selection more strict:
- /man/unknown-hash would previously ignore the hash and just select
  whatever man page. Now it results in a 404.
- Same with /man.unknown-section
- /man.section/hash is now disallowed, it's either /man.section or
  /man/hash.

1) Note that all possible man pages are currently still loaded into Perl
anyway, because the ugly navigation menu on the right needs them. I plan
to revamp that entire menu to be more efficient and usable.

2) Initially I wrote the SQL implementation in a similar fashion to the
Perl implementation, and ended up with the same bug. I wasted more than
a day before I finally got to the current CTE query.
2016-10-09 09:10:40 +02:00
Yorhel
ed00c5fd46 Add FreeBSD 9.3 2016-10-08 00:18:08 +02:00
Yorhel
72e26a0614 Improved layout of package page 2016-10-06 14:43:07 +02:00
Yorhel
d2d48920f4 SQL: Add ON DELETE clause to foreign keys
This makes it easier to remove a complete package or system when
debugging.
2016-10-06 09:52:20 +02:00
Yorhel
488b88c3cf Remove dead linuxmanpages.net link 2016-10-03 20:28:39 +02:00
Yorhel
7e17853326 Remove HTML-formatted man pages 2016-10-03 20:04:23 +02:00
Yorhel
aa94548c71 Don't substr() section names in man page URLs
And improve man page lookup on "/man.section". If "man.section" itself
is a man page, then we display that. If it's not, we look for a man page
named "man" in section "section". (And if that section isn't available
we just show whatever the preferred version is for "man").

This fixes some occasional 404 links from the search results.
2016-10-03 19:36:37 +02:00
Yorhel
1f5418a10e CSS: Fix weird layout of systems listing on homepage 2016-10-02 20:23:50 +02:00
Yorhel
03d278e4ff Normalize package info tables + change browsing URLs
This splits the 'package' table into 'packages' and 'package_versions',
which should improve performance in some cases and simplify some future
queries.

Previously it wasn't very well defined whether packages were uniquely
identified by (system, name) or by (system, category, name). This is now
normalized to the latter form. This required changes to the package URLs
to include the category.
2016-10-02 20:07:57 +02:00
Yorhel
26aefaebcd Add Ubuntu Xenial 2016-06-26 21:39:31 +02:00
Yorhel
0c6964d0e3 Minor cleanup 2016-06-26 11:33:16 +02:00
Yorhel
f7f79ad80d Don't sync old deb repos + add Ubuntu wily 2016-03-05 08:06:23 +01:00
Yorhel
8808b538f5 Add source page file name + fix handling of some odd suffixes 2015-12-19 11:47:42 +01:00