[comment {--- punk::docgen generated from inline doctools comments ---}]
[comment {--- punk::docgen DO NOT EDIT DOCS HERE UNLESS YOU REMOVE THESE COMMENT LINES ---}]
[comment {--- punk::docgen overwrites this file ---}]
[manpage_begin punkshell_module_punk::lib 0 0.1.2]
[copyright "2024"]
[titledesc {punk general utility functions}]  [comment {-- Name section and table of contents description --}]
[moddesc {punk library}]             [comment {-- Description at end of page heading --}]
[require punk::lib]
[keywords module utility lib]
[description]
[para]This is a set of utility functions that are commonly used across punk modules or are just considered to be general-purpose functions.
[para]The base set includes string and math functions but has no specific theme
[section Overview]
[para] overview of punk::lib
[subsection Concepts]
[para]The punk::lib modules should have no strong dependencies other than Tcl
[para]Dependendencies that only affect display or additional functionality may be included - but should fail gracefully if not present, and only when a function is called that uses one of these soft dependencies.
[para]This requirement for no strong dependencies, means that many utility functions that might otherwise seem worthy of inclusion here are not present.
[subsection dependencies]
[para] packages used by punk::lib
[list_begin itemized]
[item] [package {Tcl 8.6-}]
[item] [package {punk::args}]
[list_end]
[section API]
[subsection {Namespace punk::lib::compat}]
[para] compatibility functions for features that may not be available in earlier Tcl versions
[para] These are generally 'forward compatibility' functions ie allowing earlier versions to use later features/idioms by using a Tcl-only version of a missing builtin.
[para] Such Tcl-only versions will inevitably be less performant - perhaps significantly so.
[list_begin definitions]
[call [fun lremove] [arg list] [opt {index ...}]]
[para] Forwards compatible lremove for versions 8.6 or less to support equivalent 8.7 lremove
[call [fun lpop] [arg listvar] [opt {index}]]
[para] Forwards compatible lpop for versions 8.6 or less to support equivalent 8.7 lpop
[list_end] [comment {--- end definitions namespace punk::lib::compat ---}]
[subsection {Namespace punk::lib}]
[para] Core API functions for punk::lib
[list_begin definitions]
[call [fun invoke] [arg command]]
[para]Invoke an external command (using tcl open command) capturing stdout,stderr and the exitcode
[example {
    set script {
       puts stdout {hello on stdout}
       puts stderr {hello on stderr}
       exit 42
    }
    invoke [list tclsh <<$script]
}]
[call [fun lindex_resolve] [arg len] [arg index]]
[para]Resolve an index which may be of the forms accepted by Tcl list commands such as end-2 or 2+2 to the actual integer index for the supplied list/string length
[para]Users may define procs which accept a list/string index and wish to accept the forms understood by Tcl.
[para]This means the proc may be called with something like $x+2  end-$y  etc
[para]Sometimes the actual integer index is desired.
[para]We want to resolve the index used, without passing arbitrary expressions into the 'expr' function - which could have security risks.
[para]lindex_resolve will parse the index expression and return:
[para] a) -3 if the supplied index expression is below the lower bound for the supplied list. (< 0)
[para] b) -2 if the supplied index expression is above the upper bound for the supplied list. (> end)
[para] We don't return -1 - as the similar function lindex_resolve_basic uses this to denote out of range at either end of the list/string
[para]Otherwise it will return an integer corresponding to the position in the list.
[para]This is in stark contrast to Tcl list function indices which will return empty strings for out of bounds indices, or in the case of lrange, return results anyway.
[para]Like Tcl list commands - it will produce an error if the form of the index is not acceptable
[para]For empty lists, end and end+x indices are considered to be out of bounds on the upper side - thus returning -2
[call [fun lindex_resolve_basic] [arg len] [arg index]]
[para] Accepts index of the forms accepted by Tcl's list commands. (e.g compound indices such as 3+1 end-2)
[para] returns -1 for out of range at either end, or a valid integer index
[para] Unlike lindex_resolve; lindex_resolve_basic can't determine if an out of range index was out of range at the lower or upper bound
[para] This is only likely to be faster than average over lindex_resolve for small lists and for Tcl which has the builtin lseq command
[para] The performance advantage is more likely to be present when using compound indexes such as $x+1 or end-1
[para] For pure integer indices the performance should be equivalent
[call [fun K] [arg x] [arg y]]
[para]The K-combinator function - returns the first argument, x and discards y
[para]see [uri https://wiki.tcl-lang.org/page/K]
[para]It is used in cases where command-substitution at the calling-point performs some desired effect.
[call [fun is_utf8_multibyteprefix] [arg str]]
[para] Returns a boolean if str is potentially a prefix for a multibyte utf-8 character
[para] ie - tests if it is possible that appending more data will result in a utf-8 codepoint
[para] Will return false for an already complete utf-8 codepoint
[para] It is assumed the incomplete sequence is at the beginning of the bytes argument
[para] Suitable input for this might be from the unreturned tail portion of get_utf8_leading $testbytes
[para] e.g using: set head [lb]get_utf8_leading $testbytes[rb] ; set tail [lb]string range $testbytes [lb]string length $head[rb] end[rb]
[call [fun is_utf8_single] [arg 1234bytes]]
[para] Tests input of 1,2,3 or 4 bytes and responds with a boolean indicating if it is a valid utf-8 character (codepoint)
[call [fun get_utf8_leading] [arg rawbytes]]
[para] return the leading portion of rawbytes that is a valid utf8 sequence.
[para] This will stop at the point at which the bytes can't be interpreted as a complete utf-8 codepoint
[para] e.g It will not return the first byte or 2 of a 3-byte utf-8 character if the last byte is missing, and will return only the valid utf-8 string from before the first byte of the incomplete character.
[para] It will also only return the prefix before any bytes that cannot be part of a utf-8 sequence at all.
[para] Note that while this will return valid utf8 - it has no knowledge of grapheme clusters or diacritics
[para] This means if it is being used to process bytes split at some arbitrary point - the trailing data that isn't returned could be part of a grapheme cluster that belongs with the last character of the leading string already returned
[para] The utf-8 BOM \xEF\xBB\xBF is a valid UTF8 3-byte sequence and so can also be returned as part of the leading utf8 bytes
[call [fun hex2dec] [opt {option value...}] [arg list_largeHex]]
[para]Convert a list of (possibly large) unprefixed hex strings to their decimal values
[para]hex2dec accepts and ignores internal underscores in the same manner as Tcl 8.7+ numbers e.g  hex2dec FF_FF returns 65535
[para]Leading and trailing underscores are ignored as a matter of implementation convenience - but this shouldn't be relied upon.
[para]Leading or trailing whitespace in each list member is allowed e.g hex2dec " F" returns 15
[para]Internal whitespace e.g "F F" is not permitted - but a completely empty element "" is allowed and will return 0
[call [fun dex2hex] [opt {option value...}] [arg list_decimals]]
[para]Convert a list of decimal integers to a list of hex values
[para] -width <int> can be used to make each hex value at least int characters wide, with leading zeroes.
[para] -case upper|lower determines the case of the hex letters in the output
[call [fun log2] [arg x]]
[para]log base2 of x
[para]This uses a 'live' proc body - the divisor for the change of base is computed once at definition time
[para](courtesy of RS [uri https://wiki.tcl-lang.org/page/Additional+math+functions])
[call [fun logbase] [arg b] [arg x]]
[para]log base b of x
[para]This function uses expr's natural log and the change of base division.
[para]This means for example that we can get results like: logbase 10 1000 = 2.9999999999999996
[para]Use expr's log10() function or tcl::mathfunc::log10 for base 10
[call [fun factors] [arg x]]
[para]Return a sorted list of the positive factors of x where x > 0
[para]For x = 0 we return only 0 and 1 as technically any number divides zero and there are an infinite number of factors. (including zero itself in this context)*
[para]This is a simple brute-force implementation that iterates all numbers below the square root of x to check the factors
[para]Because the implementation is so simple - the performance is very reasonable for numbers below at least a few 10's of millions
[para]See tcllib math::numtheory::factors for a more complex implementation - which seems to be slower for 'small' numbers
[para]Comparisons were done with some numbers below 17 digits long
[para]For seriously big numbers - this simple algorithm would no doubt be outperformed by more complex algorithms.
[para]The numtheory library stores some data about primes etc with each call - so may become faster when being used on more numbers
but has the disadvantage of being slower for 'small' numbers and using more memory.
[para]If the largest factor below x is needed - the greatestOddFactorBelow and GreatestFactorBelow functions are a faster way to get there than computing the whole list, even for small values of x
[para]* Taking x=0; Notion of x being divisible by integer y being: There exists an integer p such that x = py
[para]  In other mathematical contexts zero may be considered not to divide anything.
[call [fun oddFactors] [arg x]]
[para]Return a list of odd integer factors of x, sorted in ascending order
[call [fun greatestFactorBelow] [arg x]]
[para]Return the largest factor of x excluding itself
[para]factor functions can be useful for console layout calculations
[para]See Tcllib math::numtheory for more extensive implementations
[call [fun greatestOddFactorBelow] [arg x]]
[para]Return the largest odd integer factor of x excluding x itself
[call [fun greatestOddFactor] [arg x]]
[para]Return the largest odd integer factor of x
[para]For an odd value of x - this will always return x
[call [fun gcd] [arg n] [arg m]]
[para]Return the greatest common divisor of m and n
[para]Straight from Lars Hellström's math::numtheory library  in Tcllib
[para]Graphical use:
[para]An a by b rectangle can be covered with square tiles of side-length c,
[para]only if c is a common divisor of a and b
[call [fun gcd] [arg n] [arg m]]
[para]Return the lowest common multiple of m and n
[para]Straight from Lars Hellström's math::numtheory library  in Tcllib
[para]
[call [fun commonDivisors] [arg x] [arg y]]
[para]Return a list of all the common factors of x and y
[para](equivalent to factors of their gcd)
[call [fun hasglobs] [arg str]]
[para]Return a boolean indicating whether str contains any of the glob characters: * ? [lb] [rb]
[para]hasglobs uses append to preserve Tcls internal representation for str - so it should help avoid shimmering in the few cases where this may matter.
[call [fun trimzero] [arg number]]
[para]Return number with left-hand-side zeros trimmed off - unless all zero
[para]If number is all zero - a single 0 is returned
[call [fun substring_count] [arg str] [arg substring]]
[para]Search str and return number of occurrences of substring
[call [fun dict_merge_ordered] [arg defaults] [arg main]]
[para]The standard dict merge accepts multiple dicts with values from dicts to the right (2nd argument) taking precedence.
[para]When merging with a dict of default values - this means that any default key/vals that weren't in the main dict appear in the output before the main data.
[para]This function merges the two dicts whilst maintaining the key order of main followed by defaults.
[call [fun askuser] [arg question]]
[para]A basic utility to read an answer from stdin
[para]The prompt is written to the terminal and then it waits for a user to type something
[para]stdin is temporarily configured to blocking and then put back in its original state in case it wasn't already so.
[para]If the terminal is using punk::console and is in raw mode - the terminal will temporarily be put in line mode.
[para](Generic terminal raw vs linemode detection not yet present)
[para]The user must hit enter to submit the response
[para]The return value is the string if any that was typed prior to hitting enter.
[para]The question argument can be manually colourised using the various punk::ansi funcitons
[example_begin]
   set answer [lb]punk::lib::askuser "[lb]a+ green bold[rb]Do you want to proceed? (Y|N)[lb]a[rb]"[rb]
   if {[lb]string match y* [lb]string tolower $answer[rb][rb]} {
       puts "Proceeding"
   } else {
       puts "Cancelled by user"
   }
[example_end]
[call [fun linesort] [opt {sortoption ?val?...}] [arg textblock]]
[para]Sort lines in textblock
[para]Returns another textblock with lines sorted
[para]options are flags as accepted by lsort ie -ascii -command -decreasing -dictionary -index -indices -integer -nocase -real -stride -unique
[call [fun list_as_lines] [opt {-joinchar char}] [arg linelist]]
[para]This simply joins the elements of the list with -joinchar
[para]It is mainly intended for use in pipelines where the primary argument comes at the end - but it can also be used as a general replacement for join $lines <le>
[para]The sister function lines_as_list takes a block of text and splits it into lines - but with more options related to trimming the block and/or each line.
[call [fun lines_as_list] [opt {option value ...}] [arg text]]
[para]Returns a list of possibly trimmed lines depeding on options
[para]The concept of lines is raw lines from splitting on newline after crlf is mapped to lf
[para]- not console lines which may be entirely different due to control characters such as vertical tabs or ANSI movements
[list_end] [comment {--- end definitions namespace punk::lib ---}]
[section Internal]
[subsection {Namespace punk::lib::system}]
[para] Internal functions that are not part of the API
[list_begin definitions]
[list_end] [comment {--- end definitions namespace punk::lib::system ---}]
[manpage_end]