Commit graph

79 commits

Author SHA1 Message Date
Yorhel
f8b0043e22 MultipartFormData: Bunch of parser fixes 2025-04-29 09:14:44 +02:00
Yorhel
d0c5397e2d json_parse()/pgtypes: Fix accidental creation of read-only array/hash values
&PL_sv_* shouldn't be used when constructing arrays or hashes in this
context.
2025-04-28 10:23:51 +02:00
Yorhel
753cac615a Validate: Improved + extendable error message formatting
Very much needed for VNDB's advanced search validation.
Also completely undocumented.
2025-04-26 15:56:12 +02:00
Yorhel
5f8809d052 FU::Util::query_decode(): Properly handle empty "&"-parts 2025-04-25 17:07:56 +02:00
Yorhel
0cd947c545 FastCGI: Ignore HTTP_CONTENT_(TYPE|LENGTH)
The non-HTTP_ versions of these are authoritative, Also fixes a memory
leak when both the HTTP_ and non-HTTP_ versions are included.
2025-04-25 09:34:22 +02:00
Yorhel
ab168bd952 Pg: Don't return self on / cache() or text_*() methods
This is much too easy of a footgun:

  $db->cache->q(...);

Enabled cache on the $db object, not just for the given query.
2025-04-24 07:23:03 +02:00
Yorhel
8b807e6dcf Validate: Add empty() and coerce() methods
Implementing the undocumented coerce_for_json() method from TUWF and
elm_empty() from VNDB.
2025-04-19 12:36:28 +02:00
Yorhel
b3281924d1 Pg: Add escape_literal() and escape_identifier()
Didn't expect I'd ever need these, but they're useful for generating SQL
scripts.
2025-04-07 13:45:33 +02:00
Yorhel
13661b46f9 Validate: Normalize num/int/uint to Perl numeric types + add 64bit limit to int/uint
Normalization may be undone by later validations, but this should work
in most cases.
2025-03-30 13:55:23 +02:00
Yorhel
6159b33950 FU::Util: Add brotli_compress() and use it for FU output compression
Seems to compresses and perform better than libdeflate at level 6, so
certainly worth using.
2025-03-19 10:12:20 +01:00
Yorhel
bc33fe53f0 FU::Util: Add gzip_compress() wrapper for libdeflate/zlib-ng/zlib
And use it for automatic output compression in FU, as (potentially)
faster alternative to Compress::Raw::Zlib.

Was also planning to maybe add support for Zstd or Brotli, but given the
performance of libdeflate, I'm not sure that's really necessary. Brotli
does tend to do a better job at compressing HTML, though.
2025-03-18 16:59:43 +01:00
Yorhel
a7bfe146b1 Validate: Fix referencing & merging already compiled schemas 2025-03-17 12:50:24 +01:00
Yorhel
3382deba9a Validate: Rename "scalar" to "accept_scalar" and add "accept_array" 2025-03-16 15:39:10 +01:00
Yorhel
f8fe53cba9 json_format: Add html_safe option 2025-03-16 15:03:32 +01:00
Yorhel
3fad7feec3 Validate: Rename "values"->"elems", repurpose "values" to validate hash values
I'm breaking stuff left and right while I still can.

Idea: "key_names" validation?

Idea: "tuple" validation that works like "keys" but for arrays.
  (i.e. { tuple => { $index => $schema } }, could make "missing" and
  "unknown" work for arrays, too)
2025-03-14 16:52:33 +01:00
Yorhel
fa24ca53e3 Validate: improved arrayref-schema semantics
This allows all built-in options to be duplicated inside a single
schema, the semantics of which are the same as the kind of merging
done as part of inheriting options from custom validations.

This also causes all 'keys' and 'values' validation schemas to be
merged, which changes error messages a bit but is great for
introspection. Probably slightly improves performance as well.
2025-03-14 14:28:28 +01:00
Yorhel
f248a33c1c Validate: Allow multiple func validations in arrayref schema 2025-03-14 09:25:56 +01:00
Yorhel
1363e11269 Validate: allow array schemas + defer known_keys hash creation
Doesn't allow multiple 'func' options yet, needs work.
2025-03-14 07:28:11 +01:00
Yorhel
d9d2ad0434 Pg: Add COPY support 2025-03-10 12:32:14 +01:00
Yorhel
dc752e2a23 Pg: Support dynamic-oid types + vndbtag/vndbid 2025-03-09 10:23:48 +01:00
Yorhel
17176738a0 FU: Support multipart file uploads + some doc fixes
API is not super convenient and implementation is lousy, but uploading
files is not a super common operation so that should be fine.

At least it supports large files with only a single in-memory copy.
2025-03-08 14:06:10 +01:00
Yorhel
e5755ddd80 Validate: Human-readable error messages 2025-03-06 10:16:14 +01:00
Yorhel
e4b6b77e1b Validate: rename rmwhitespace to trim and use builtin::trim() 2025-03-05 15:39:49 +01:00
Yorhel
cbebc3a21e Validate: Rework API, ->validate() now throws error instead of result object
This is a slight simplification and removes the need to pass around
partially normalized data. I've never found a use for the unsafe_data()
method.
2025-03-05 15:35:23 +01:00
Yorhel
f09a103c53 Some test portability fixes again + minor changes 2025-03-02 10:10:35 +01:00
Yorhel
15954f4ad5 Pg: Add "$hex" pseudo-type 2025-02-28 13:49:07 +01:00
Yorhel
baf0f90bd5 Pg: Add "time" type, change default format of "date" type but support "$date_str"
Using a consistent numeric interpretation for timestamps, dates and
times simplifies a bunch of operations. It certainly simplifies
conversion between the Postgres formats.
2025-02-28 13:03:16 +01:00
Yorhel
0734bc4991 Pg: Skip query preparation when we can 2025-02-28 12:33:30 +01:00
Yorhel
4686097d00 Pg: Support custom type overrides with callbacks 2025-02-28 11:23:42 +01:00
Yorhel
327fd9ea50 Pg: Support type override configuration 2025-02-27 18:24:14 +01:00
Yorhel
de36b90cde Fixes for longdouble perl builds
Mostly test fixes, but the Pg 'timestamp' type did have a small
conversion bug.
2025-02-27 09:35:15 +01:00
Yorhel
7b0ba45346 Docs, compatibility fixes again, 0.1 release 2025-02-25 17:02:33 +01:00
Yorhel
69262992ca FU::Validate: Add module + integrate with FU
Copied from TUWF::Validate with a few small changes. I have a few more
features planned, but let's see how this goes first.

It's been an incredibly useful module in the past, I'm not sure right
now if I had ideas for potential improvements at some point, will need
to check notes.
2025-02-25 14:31:38 +01:00
Yorhel
c7a3415485 Util: Add to_bool() and use it for JSON, Pg & query encoding
To improve interop with legacy modules.
2025-02-25 09:33:33 +01:00
Yorhel
06e2f950fe Add fu->redirect, change $st->row behavior on 0 results, minor fixes
And with this, I have a working rewrite of the manned.org backend into
FU. \o/

The $st->row methods are very useful even for queries that may not
return anything, so their old behavior was unhelpful. Interestingly
enough, the error-on-multiple-rows did catch an actual bug in
Manned.org, so I'm keeping that behavior.
2025-02-24 15:55:27 +01:00
Yorhel
fbbaa23842 pg: Add date type & httpdate tests
...I was hoping not to have to implement the date type, because date
conversions suck, but it turns out manned.org actually needs it.
(Only to then convert it into a Unix timestamp again, hmm, maybe this
string conversion isn't useful at all?)
2025-02-24 11:54:37 +01:00
Yorhel
18e642290d Some fixes and framework docs 2025-02-23 14:05:43 +01:00
Yorhel
b2d676b1ed pg: Add query tracing & prepare/execute time measurements
What I'd really like, in addition to this, is a way to extract a query
from an $st object that can be run in the psql CLI. VNDB has a debugging
feature for that, but it's less trivial to make that work with binary
query parameters.
2025-02-22 15:15:16 +01:00
Yorhel
6f1583ddad pg: timestamp(tz) types + more docs 2025-02-21 12:58:24 +01:00
Yorhel
8036b8f0bf SQL: Doc and test adjustments 2025-02-21 10:22:33 +01:00
Yorhel
145d086bea Add FU::SQL
Somewhat experimental, need to test this approach on a real site.
Looking pretty promising so far, though.
2025-02-20 15:45:55 +01:00
Yorhel
9014e2900c Add FU::XMLWriter
And remove UTF-8 check in JSON writer. It honestly feels kind of silly
to do that validation there while I've never done similar validations in
any other output routines - including this XML writer.

FU::XMLWriter is a copy of TUWF::XMLXS with a bunch of improvements
applied: now uses refcounts to determine the current output instance,
auto-generates XS functions and has faster escaped string output -
inspired by the JSON writer.

TODO:
- Integrate into FU
- Do something with bool attribute values
- Benchmarks
- Should $content be optional for all tags? The reason they weren't in
  TUWF::XMLXS is because TUWF::XML supports opening tags without closing
  them, but that idea turned out to suck and isn't supported anymore.

This is hopefully the last XS module for the FU framework. The only C
code being written now should be bug fixes and extending FU::Pg with
some planned features. Already ended up with more C than I had
planned...
2025-02-19 12:01:24 +01:00
Yorhel
67e6d99f01 Add query string encoding & decoding functions 2025-02-18 14:08:23 +01:00
Yorhel
90cfd66069 FU: Some FastCGI fixes; FU::Util: utf8_decode & URI escaping 2025-02-18 10:27:58 +01:00
Yorhel
d9fba4e8d8 FU: Add FastCGI support + bunch of fixes
I initially planned to only implement the bare minimum to support
FastCGI under nginx, but ended up implementing the full protocol
instead. This is more code than I had expected and the code is also less
trivial than I had hoped. Will need to do more testing, pretty sure
there's bugs left.

Also TODO: test under alternative process managers + document
FU_LISTEN_PROTO.

I've also removed the max_request_body setting, this is something that
really ought to be configured in the web server instead.
2025-02-17 15:15:38 +01:00
Yorhel
3e84a4f4d3 FU: Implement --monitor, add some docs; FU::Util: add fdpass functions 2025-02-15 15:09:56 +01:00
Yorhel
d5401674f9 pg: Minor docs + tests 2025-02-13 08:50:40 +01:00
Yorhel
867543267f Fixes for perl 5.36 with multiplicity + memleak in $st->kv methods
I always keep messing up the aTHX_ and pTHX_ stuff because my system
perl isn't built with multiplicity, and I still haven't found a
satisfactory way of finding SV leaks. Valgrind can't track those :(
2025-02-13 06:26:58 +01:00
Yorhel
1f7e2de9a0 pg: Add prepared statement caching
The tests are not as thourough as I would like. There's many ways to
mess this up.

I was initially planning to drop the ref on the prepared statement
immediately after executing the query, so that the $st object can be
kept around for introspection without consuming excess resources.
Unfortunately, PQcopyResult does not copy over information about bind
parameters, so we need another way to keep that information alive. I
ended up going for the simple solution: keep the ref on the prepared
statement...
2025-02-12 17:19:20 +01:00
Yorhel
87d99e412b pg: Add support for record/composite types
This is simply magical. \o/

Vendored in khashl.h. I wouldn't have used it if this were the only
place where I'd need a custom hash table, but it should come in handy
for other tasks as well, especially when I get to implementing an LRU
for prepared statement caching.

(Can all be done with Perl HV's, but they're less efficient and more
cumbersome for these tasks)
2025-02-11 16:04:10 +01:00