Commit graph

92 commits

Author SHA1 Message Date
Yorhel
a7e9fa1866 FU: Less verbose and cryptic error message on write error 2026-01-05 13:20:59 +01:00
Yorhel
48fe393d5f FastCGI: Improve handling of EPIPE while writing response
That would previously result in the worker getting killed with SIGPIPE.
Which works, but we can also recover from that error without restarting
the process.
2026-01-05 08:57:50 +01:00
Yorhel
8140fefbca FU::Pg: Rename q() and Q() to sql() and SQL()
Because this easily confuses syntax highlighters and some humans with
the q// string syntax. Also for consistency with the 'fu->sql()'
aliases.

The old names are still available as alias.
2025-12-04 14:16:37 +01:00
Yorhel
a8ac435f85 Move control character checking to FU::Validate, deprecate FU::Util::utf8_decode()
URI, JSON and formdata decoding no longer checks for control characters,
but FU::Validate now rejects control characters by default. This
decouples semantic validation from format parsing and gives better
control over when control characters are allowed.
2025-08-22 10:03:06 +02:00
Yorhel
2e9a40da69 More strict UTF-8 validation on input 2025-08-22 09:21:06 +02:00
Yorhel
a7868f74bf Pg: Be more strict with boolean bind parameters
Reason for this is that, with FU::SQL, it's possible to accidentally
introduce a bind parameter when a WHERE clause was intended (i.e.
"WHERE $1"). That's pretty bad, but can easily be caught by simply not
accepting *every* possible value as boolean.
2025-06-12 16:45:09 +02:00
Yorhel
55baa6c9a6 json_parse(): Disallow control characters in strings by default
Deviating from the standard, but more consistent other FU functions.
2025-06-04 18:48:08 +02:00
Yorhel
a43dc70ff9 XMLWriter: Throw error when stringifying a bare reference
I can't think of a use case where Perl's default ref stringification is
something you actually want when writing XML/HTML - this pretty much
always points to a bug. One that I seem to be prone to making...
2025-06-02 09:00:06 +02:00
Yorhel
fd8332601b t/pgconnect: Fix ref leak in test
Apparently 'my sub' captured the $conn variable and held a ref on it
even beyond the parent sub scope. 'my $x = sub {}' doesn't do that.

Getting the ref counts right is important here for the last test to work.

(Found while I was inspecting the refcount effects of the new ->conn()
methods with Devel::Peek)
2025-05-22 09:56:11 +02:00
Yorhel
2083ab2a6f Pg: Set appropriate quote_identifier for $conn->Q() 2025-05-22 09:53:41 +02:00
Yorhel
81a3d3c608 SQL: Add IDENT() and quote_identifier options
Turns out VNDB has a few places where request data is directly used for
column names in VALUES/SET/WHERE clauses. These are already restricted
to known strings through the use of FU::Validate, but an extra layer of
protection seems warranted here.
2025-05-22 09:22:07 +02:00
Yorhel
76f55f277b Pg: Add text2bin() and bin2text() conversion methods 2025-04-30 20:02:20 +02:00
Yorhel
beeefcf337 Pg: Add perl2bin() and bin2perl() conversion methods 2025-04-30 17:07:44 +02:00
Yorhel
f8b0043e22 MultipartFormData: Bunch of parser fixes 2025-04-29 09:14:44 +02:00
Yorhel
d0c5397e2d json_parse()/pgtypes: Fix accidental creation of read-only array/hash values
&PL_sv_* shouldn't be used when constructing arrays or hashes in this
context.
2025-04-28 10:23:51 +02:00
Yorhel
753cac615a Validate: Improved + extendable error message formatting
Very much needed for VNDB's advanced search validation.
Also completely undocumented.
2025-04-26 15:56:12 +02:00
Yorhel
5f8809d052 FU::Util::query_decode(): Properly handle empty "&"-parts 2025-04-25 17:07:56 +02:00
Yorhel
0cd947c545 FastCGI: Ignore HTTP_CONTENT_(TYPE|LENGTH)
The non-HTTP_ versions of these are authoritative, Also fixes a memory
leak when both the HTTP_ and non-HTTP_ versions are included.
2025-04-25 09:34:22 +02:00
Yorhel
ab168bd952 Pg: Don't return self on / cache() or text_*() methods
This is much too easy of a footgun:

  $db->cache->q(...);

Enabled cache on the $db object, not just for the given query.
2025-04-24 07:23:03 +02:00
Yorhel
8b807e6dcf Validate: Add empty() and coerce() methods
Implementing the undocumented coerce_for_json() method from TUWF and
elm_empty() from VNDB.
2025-04-19 12:36:28 +02:00
Yorhel
b3281924d1 Pg: Add escape_literal() and escape_identifier()
Didn't expect I'd ever need these, but they're useful for generating SQL
scripts.
2025-04-07 13:45:33 +02:00
Yorhel
13661b46f9 Validate: Normalize num/int/uint to Perl numeric types + add 64bit limit to int/uint
Normalization may be undone by later validations, but this should work
in most cases.
2025-03-30 13:55:23 +02:00
Yorhel
6159b33950 FU::Util: Add brotli_compress() and use it for FU output compression
Seems to compresses and perform better than libdeflate at level 6, so
certainly worth using.
2025-03-19 10:12:20 +01:00
Yorhel
bc33fe53f0 FU::Util: Add gzip_compress() wrapper for libdeflate/zlib-ng/zlib
And use it for automatic output compression in FU, as (potentially)
faster alternative to Compress::Raw::Zlib.

Was also planning to maybe add support for Zstd or Brotli, but given the
performance of libdeflate, I'm not sure that's really necessary. Brotli
does tend to do a better job at compressing HTML, though.
2025-03-18 16:59:43 +01:00
Yorhel
a7bfe146b1 Validate: Fix referencing & merging already compiled schemas 2025-03-17 12:50:24 +01:00
Yorhel
3382deba9a Validate: Rename "scalar" to "accept_scalar" and add "accept_array" 2025-03-16 15:39:10 +01:00
Yorhel
f8fe53cba9 json_format: Add html_safe option 2025-03-16 15:03:32 +01:00
Yorhel
3fad7feec3 Validate: Rename "values"->"elems", repurpose "values" to validate hash values
I'm breaking stuff left and right while I still can.

Idea: "key_names" validation?

Idea: "tuple" validation that works like "keys" but for arrays.
  (i.e. { tuple => { $index => $schema } }, could make "missing" and
  "unknown" work for arrays, too)
2025-03-14 16:52:33 +01:00
Yorhel
fa24ca53e3 Validate: improved arrayref-schema semantics
This allows all built-in options to be duplicated inside a single
schema, the semantics of which are the same as the kind of merging
done as part of inheriting options from custom validations.

This also causes all 'keys' and 'values' validation schemas to be
merged, which changes error messages a bit but is great for
introspection. Probably slightly improves performance as well.
2025-03-14 14:28:28 +01:00
Yorhel
f248a33c1c Validate: Allow multiple func validations in arrayref schema 2025-03-14 09:25:56 +01:00
Yorhel
1363e11269 Validate: allow array schemas + defer known_keys hash creation
Doesn't allow multiple 'func' options yet, needs work.
2025-03-14 07:28:11 +01:00
Yorhel
d9d2ad0434 Pg: Add COPY support 2025-03-10 12:32:14 +01:00
Yorhel
dc752e2a23 Pg: Support dynamic-oid types + vndbtag/vndbid 2025-03-09 10:23:48 +01:00
Yorhel
17176738a0 FU: Support multipart file uploads + some doc fixes
API is not super convenient and implementation is lousy, but uploading
files is not a super common operation so that should be fine.

At least it supports large files with only a single in-memory copy.
2025-03-08 14:06:10 +01:00
Yorhel
e5755ddd80 Validate: Human-readable error messages 2025-03-06 10:16:14 +01:00
Yorhel
e4b6b77e1b Validate: rename rmwhitespace to trim and use builtin::trim() 2025-03-05 15:39:49 +01:00
Yorhel
cbebc3a21e Validate: Rework API, ->validate() now throws error instead of result object
This is a slight simplification and removes the need to pass around
partially normalized data. I've never found a use for the unsafe_data()
method.
2025-03-05 15:35:23 +01:00
Yorhel
f09a103c53 Some test portability fixes again + minor changes 2025-03-02 10:10:35 +01:00
Yorhel
15954f4ad5 Pg: Add "$hex" pseudo-type 2025-02-28 13:49:07 +01:00
Yorhel
baf0f90bd5 Pg: Add "time" type, change default format of "date" type but support "$date_str"
Using a consistent numeric interpretation for timestamps, dates and
times simplifies a bunch of operations. It certainly simplifies
conversion between the Postgres formats.
2025-02-28 13:03:16 +01:00
Yorhel
0734bc4991 Pg: Skip query preparation when we can 2025-02-28 12:33:30 +01:00
Yorhel
4686097d00 Pg: Support custom type overrides with callbacks 2025-02-28 11:23:42 +01:00
Yorhel
327fd9ea50 Pg: Support type override configuration 2025-02-27 18:24:14 +01:00
Yorhel
de36b90cde Fixes for longdouble perl builds
Mostly test fixes, but the Pg 'timestamp' type did have a small
conversion bug.
2025-02-27 09:35:15 +01:00
Yorhel
7b0ba45346 Docs, compatibility fixes again, 0.1 release 2025-02-25 17:02:33 +01:00
Yorhel
69262992ca FU::Validate: Add module + integrate with FU
Copied from TUWF::Validate with a few small changes. I have a few more
features planned, but let's see how this goes first.

It's been an incredibly useful module in the past, I'm not sure right
now if I had ideas for potential improvements at some point, will need
to check notes.
2025-02-25 14:31:38 +01:00
Yorhel
c7a3415485 Util: Add to_bool() and use it for JSON, Pg & query encoding
To improve interop with legacy modules.
2025-02-25 09:33:33 +01:00
Yorhel
06e2f950fe Add fu->redirect, change $st->row behavior on 0 results, minor fixes
And with this, I have a working rewrite of the manned.org backend into
FU. \o/

The $st->row methods are very useful even for queries that may not
return anything, so their old behavior was unhelpful. Interestingly
enough, the error-on-multiple-rows did catch an actual bug in
Manned.org, so I'm keeping that behavior.
2025-02-24 15:55:27 +01:00
Yorhel
fbbaa23842 pg: Add date type & httpdate tests
...I was hoping not to have to implement the date type, because date
conversions suck, but it turns out manned.org actually needs it.
(Only to then convert it into a Unix timestamp again, hmm, maybe this
string conversion isn't useful at all?)
2025-02-24 11:54:37 +01:00
Yorhel
18e642290d Some fixes and framework docs 2025-02-23 14:05:43 +01:00