Download the PHP package nguyenanhung/htmlawed without Composer

/ htmLawed_README.txt, 5 November 2022 htmLawed 1.2.10 Copyright Santosh Patnaik Dual licensed with LGPL 3 and GPL 2+ A PHP Labware internal utility - https://bioinformatics.org/phplabware/internal_utilities/htmLawed /

== Content =========================================================

1 About htmLawed 1.1 Example uses 1.2 Features 1.3 History 1.4 License & copyright 1.5 Terms used here 1.6 Availability 2 Usage 2.1 Simple 2.2 Configuring htmLawed using the '$config' argument 2.3 Extra HTML specifications using the '$spec' argument 2.4 Performance time & memory usage 2.5 Some security risks to keep in mind 2.6 Use with 'kses()' code 2.7 Tolerance for ill-written HTML 2.8 Limitations & work-arounds 2.9 Examples of usage 3 Details 3.1 Invalid/dangerous characters 3.2 Character references/entities 3.3 HTML elements 3.3.1 HTML comments & 'CDATA' sections 3.3.2 Tag-transformation for better compliance with standards 3.3.3 Tag balancing & proper nesting 3.3.4 Elements requiring child elements 3.3.5 Beautify or compact HTML 3.3.6 Custom elements 3.4 Attributes 3.4.1 Auto-addition of XHTML-required attributes 3.4.2 Duplicate/invalid 'id' values 3.4.3 URL schemes & scripts in attribute values 3.4.4 Absolute & relative URLs 3.4.5 Lower-cased, standard attribute values 3.4.6 Transformation of deprecated attributes 3.4.7 Anti-spam & 'href' 3.4.8 Inline style properties 3.4.9 Hook function for tag content 3.5 Simple configuration directive for most valid XHTML 3.6 Simple configuration directive for most safe HTML 3.7 Using a hook function 3.8 Obtaining finalized parameter values 3.9 Retaining non-HTML tags in input with mixed markup 4 Other 4.1 Support 4.2 Known issues 4.3 Change-log 4.4 Testing 4.5 Upgrade, & old versions 4.6 Comparison with 'HTMLPurifier' 4.7 Use through application plug-ins/modules 4.8 Use in non-PHP applications 4.9 Donate 4.10 Acknowledgements 5 Appendices 5.1 Characters discouraged in HTML 5.2 Valid attribute-element combinations 5.3 CSS 2.1 properties accepting URLs 5.4 Microsoft Windows 1252 character replacements 5.5 URL format 5.6 Brief on htmLawed code

== 1 About htmLawed ================================================

htmLawed is a PHP script to process text with HTML markup to make it more compliant with HTML standards and with administrative policies. It works by making HTML well-formed with balanced and properly nested tags, neutralizing code that introduces a security vulnerability or is used for cross-site scripting (XSS) attacks, allowing only specified HTML tags and attributes, and so on. Such lawing in of HTML code ensures that it is in accordance with the aesthetics, safety and usability requirements set by administrators.

htmLawed is highly customizable, and fast with low memory usage. Its free and open-source code is in one small file. It does not require extensions or libraries, and works in older versions of PHP as well. It is a good alternative to the HTML Tidy:- http://tidy.sourceforge.net application.

-- 1.1 Example uses ------------------------------------------------

• Filtering of text submitted as comments on blogs to allow only certain HTML elements

• Making RSS newsfeed items standard-compliant: often one uses an excerpt from an HTML document for the content, and with unbalanced tags, non-numerical entities, etc., such excerpts may not be XML-compliant

• Beautifying or pretty-printing HTML code

• Text processing for stricter XML standard-compliance: e.g., to have lowercased 'x' in hexadecimal numeric entities becomes necessary if an HTML document with MathML content needs to be served as 'application/xml'

• Scraping text from web-pages

• Transforming an HTML element to another

-- 1.2 Features ---------------------------------------------------o

Key: '*' security feature, '^' standard compliance, '~' requires setting right options

htmLawed:

• makes input more secure and standard-compliant for HTML as well as generic XML documents ^
• supports markup for HTML 5, custom elements, and microdata, ARIA, Ruby, custom attributes, etc. ^
• can beautify or compact HTML ~
• works with input of almost any character encoding and does not affect it

• has good tolerance for ill-written HTML

• can enforce restricted use of elements *~
• ensures proper closure of empty elements like 'img' ^
• transforms deprecated elements like 'font' ^~
• can permit HTML comments and CDATA sections ^~

• can permit all elements, including 'script', 'object' and 'form' ~

• can restrict attributes by element ^~
• removes invalid attributes ^
• lower-cases element and attribute names ^
• provides required attributes, like 'alt' for 'image' ^
• transforms deprecated attributes ^~
• ensures attributes are declared only once ^

• permits custom, non-standard attributes as well as custom rules for standard attributes ~

• declares value for empty (minimized or boolean) attributes like 'checked' ^
• checks for potentially dangerous attribute values *~
• ensures unique 'id' attribute values ^~
• double-quotes attribute values ^

• lower-cases standard attribute values like 'password' ^

• can restrict URL protocol/scheme by attribute *~

• can disable dynamic expressions in 'style' values *~

• neutralizes invalid named character entities ^
• converts hexadecimal numeric entities to decimal ones, or vice versa ^~

• converts named entities to numeric ones for generic XML use ^~

• removes null characters *
• neutralizes potentially dangerous proprietary Netscape Javascript entities *

• replaces potentially dangerous soft-hyphen character in URL-accepting attribute values with spaces *

• removes common invalid characters not allowed in HTML or XML ^
• replaces characters from Microsoft applications like 'Word' that are discouraged in HTML or XML ^~
• neutralize entities for characters invalid or discouraged in HTML or XML ^

• appropriately neutralize '<', '&', '"', and '>' characters ^*

• understands improperly spaced tag content (e.g., spread over more than a line) and properly spaces them
• attempts to balance tags for well-formedness ^~
• understands when omitable closing tags like '' are missing ^~

• attempts to permit only validly nested tags ^~
• can either remove or neutralize bad content ^~

• attempts to rectify common errors of plain-text misplacement (e.g., directly inside 'blockquote') ^~

• has optional anti-spam measures such as addition of 'rel="nofollow"' and link-disabling ~

• optionally makes relative URLs absolute, and vice versa ~

• optionally marks '&' to identify the entities for '&', '<' and '>' introduced by it ~

• allows deployment of powerful hook functions to inject HTML, consolidate 'style' attributes to 'class', finely check attribute values, etc. ~

-- 1.3 History ----------------------------------------------------o

htmLawed was created in 2007 for use with 'LabWiki', a wiki software developed at PHP Labware, as a suitable software could not be found. Existing PHP software like 'Kses' and 'HTMLPurifier' were deemed inadequate, slow, resource-intensive, or dependent on an extension or external application like 'HTML Tidy'. The core logic of htmLawed, that of identifying HTML elements and attributes, was based on the 'Kses' (version 0.2.2) HTML filter software of Ulf Harnhammar (it can still be used with code that uses 'Kses'; see section:- #2.6.). Support for HTML version 5 was added in May 2013 in a beta and in February 2017 in a production release.

See section:- #4.3 for a detailed log of changes in htmLawed over the years, and section:- #4.10 for acknowledgements.

-- 1.4 License & copyright ----------------------------------------o

htmLawed is free and open-source software, copyrighted by Santosh Patnaik, MD, PhD, and dual-licensed with LGPL version 3:- http://www.gnu.org/licenses/lgpl-3.0.txt, and GPL version 2:- http://www.gnu.org/licenses/gpl-2.0.txt (or later) licenses.

-- 1.5 Terms used here --------------------------------------------o

In this document, only HTML body-level elements are considered. htmLawed does not have support for head-level elements, 'body', and the frame-level elements, 'frameset', 'frame' and 'noframes', and these elements are ignored here.

• administrator - or admin; person setting up the code that utilizes htmLawed; also, user
• attributes - name-value pairs like 'href="http://x.com"' in opening tags
• author - see writer
• character - atomic unit of text; internally represented by a numeric code-point as specified by the encoding or charset in use
• entity - markup like '>' and ' ' used to refer to a character
• element - HTML element like 'a' and 'img'
• element content - content between the opening and closing tags of an element, like 'click' of the 'click' element
• HTML - implies XHTML unless specified otherwise
• HTML body - content in the body container of an HTML document
• input - text given to htmLawed to process
• legal – standard-compliant; also, valid
• processing - involves filtering, correction, etc., of input
• safe - absence or reduction of certain characters and HTML elements and attributes in HTML of text that can otherwise potentially, and circumstantially, expose text readers to security vulnerabilities like cross-site scripting attacks (XSS)
• scheme - a URL protocol like 'http' and 'ftp'
• specification - detailed description including rules that define HTML
• standard – widely accepted specification
• style property - terms like 'border' and 'height' for which declarations are made in values for the 'style' attribute of elements
• tag - markers like '' and '' delineating element content; the opening tag can contain attributes
• tag content - consists of tag markers '<' and '>', element names like 'div', and possibly attributes
• user - administrator
• valid - see legal
• writer - end-user like a blog commenter providing the input that is to be processed; also, author
• XHTML - XML-compliant HTML; parsing rules for XHTML are more strict than for regular HTML

-- 1.6 Availability -----------------------------------------------o

htmLawed can be downloaded for free at its website:- https://bioinformatics.org/phplabware/internal_utilities/htmLawed. Besides the 'htmLawed.php' file, the download has the htmLawed documentation (this document) in plain text:- htmLawed_README.txt and HTML:- htmLawed_README.htm formats, a script for testing:- htmLawedTest.php, and a text file for test-cases:- htmLawed_TESTCASE.txt. htmLawed can be installed with Composer, and is also available as a PHP class (OOP code) – see the website:- https://bioinformatics.org/phplabware/internal_utilities/htmLawed. Official htmLawed releases are also put up on Sourceforge:- https://sourceforge.net/projects/htmlawed/.

== 2 Usage =======================================================oo

htmLawed works in PHP version 4.4 or higher. Either 'include()' the 'htmLawed.php' file, or copy-paste the entire code.

To use with PHP 4.3, have the following code included:

if(!function_exists('ctype_digit')){
 function ctype_digit($var){
  return ((int) $var == $var);
 }
}

-- 2.1 Simple ------------------------------------------------------

The input text to be processed, '$text', is passed as an argument of type string; 'htmLawed()' returns the processed string:

$processed = htmLawed($text);

With the 'htmLawed class' (section:- #1.6), usage is:

$processed = htmLawed::hl($text);

Notes: (1) If input is from a '$_GET' or '$_POST' value, and 'magic quotes' are enabled on the PHP setup, run 'stripslashes()' on the input before passing to htmLawed. (2) htmLawed does not have support for head-level elements, 'body', and the frame-level elements, 'frameset', 'frame' and 'noframes'.

By default, htmLawed will process the text allowing all valid HTML elements/tags and commonly used URL schemes and CSS style properties. It will allow Javascript code, 'CDATA' sections and HTML comments, balance tags, and ensure proper nesting of elements. Such actions can be configured using two other optional arguments -- '$config' and '$spec':

$processed = htmLawed($text, $config, $spec); 

The '$config' and '$spec' arguments are detailed below. Some examples are shown in section:- #2.9. For maximum protection against 'XSS' and other security vulnerabilities, consider using the 'safe' parameter; see section:- #3.6.

-- 2.2 Configuring htmLawed using the '$config' argument ---------o

'$config' instructs htmLawed on how to tackle certain tasks. When '$config' is not specified, or not set as an array (e.g., '$config = 1'), htmLawed will take default actions. One or many of the task-action or parameter-value pairs can be specified in '$config' as array key-value pairs. If a parameter is not specified, htmLawed will use the default value for it, indicated further below. In PHP code, parameter values that are integers should not be quoted and should be used as numeric types (unless meant as string/text). Thus, for instance:

$config = array('comment'=>0, 'cdata'=>1, 'elements'=>'a, b, strong');
$processed = htmLawed($text, $config);

Below are the various parameters that can be specified in '$config'.

Key: '*' default, '^' different from htmLawed versions below 1.2, '~' different default when 'valid_xhtml' is set to '1' (see section:- #3.5), '"' different default when 'safe' is set to '1' (see section:- #3.6)

abs_url Make URLs absolute or relative; '$config["base_url"]' needs to be set; see section:- #3.4.4

'-1' - make relative '0' - no action * '1' - make absolute

and_mark Mark '&' characters in the original input; see section:- #3.2

anti_link_spam Anti-link-spam measure; see section:- #3.4.7

'0' - no measure taken array("regex1", "regex2") - will ensure a 'rel' attribute with 'nofollow' in its value in case the 'href' attribute value matches the regular expression pattern 'regex1', and/or will remove 'href' if its value matches the regular expression pattern 'regex2'. E.g., 'array("/./", "/://\W(?!(abc.com|xyz.org))/")'; see section:- #3.4.7 for more.

anti_mail_spam Anti-mail-spam measure; see section:- #3.4.7

'0' - no measure taken * word - '@' in mail address in 'href' attribute value is replaced with specified word

any_custom_element Permit any custom element; regardless of this setting, specific custom elements can be denied or permitted through '$config["elements"]'; see section:- #3.3.6

'0' - no '1' - yes *

balance Balance tags for well-formedness and proper nesting; see section:- #3.3.3

'0' - no '1' - yes *

base_url Base URL value that needs to be set if '$config["abs_url"]' is not '0'; see section:- #3.4.4

cdata Handling of 'CDATA' sections; see section:- #3.3.1

'0' - don't consider 'CDATA' sections as markup and proceed as if plain text " '1' - remove '2' - allow, but neutralize any '<', '>', and '&' inside by converting them to named entities '3' - allow *

clean_ms_char Replace discouraged characters introduced by Microsoft Word, etc.; see section:- #3.1

'0' - no * '1' - yes '2' - yes, but replace special single & double quotes with ordinary ones

comment Handling of HTML comments; see section:- #3.3.1

'0' - don't consider comments as markup and proceed as if plain text " '1' - remove '2' - allow, but neutralize any '<', '>', and '&' inside by converting to named entities '3' - allow *

css_expression Allow dynamic CSS expression by not removing the expression from CSS property values in 'style' attributes; see section:- #3.4.8

'0' - remove * '1' - allow

deny_attribute Denied HTML attributes; see section:- #3.4

'0' - none string - dictated by values in string 'on' - on* event attributes like 'onfocus' not allowed "

direct_nest_list Allow direct nesting of a list within another without requiring it to be a list item; see section:- #3.3.4

'0' - no * '1' - yes

elements Allowed HTML elements; see section:- #3.3

all - ^ ' -acronym -big -center -dir -font -isindex -s -strike -tt' - ~^ applet, audio, canvas, dialog, embed, iframe, object, script, and video elements not allowed - "^

hexdec_entity Allow hexadecimal numeric entities and do not convert to the more widely accepted decimal ones, or convert decimal to hexadecimal ones; see section:- #3.2

'0' - no '1' - yes * '2' - convert decimal to hexadecimal ones

hook Name of an optional hook function to alter the input string, '$config' or '$spec' before htmLawed enters the main phase of its work; see section:- #3.7

'0' - no hook function * name - name is name of the hook function

hook_tag Name of an optional hook function to alter tag content finalized by htmLawed; see section:- #3.4.9

'0' - no hook function * name - name is name of the hook function

keep_bad Neutralize bad tags by converting their '<' and '>' characters to entities, or remove them; see section:- #3.3.3

'0' - remove
'1' - neutralize both tags and element content '2' - remove tags but neutralize element content '3' and '4' - like '1' and '2' but remove if text ('pcdata') is invalid in parent element '5' and '6' * - like '3' and '4' but line-breaks, tabs and spaces are left

lc_std_val For XHTML compliance, predefined, standard attribute values, like 'get' for the 'method' attribute of 'form', must be lowercased; see section:- #3.4.5

'0' - no '1' - yes *

make_tag_strict Transform or remove these deprecated HTML elements, even if they are allowed by the admin: acronym, applet, big, center, dir, font, isindex, s, strike, tt; see section:- #3.3.2

'0' - no
'1' - yes, but leave 'applet' and 'isindex' that currently cannot be transformed *^ '2' - yes, removing 'applet' and 'isindex' elements and their contents (nested elements remain) ~^

named_entity Allow non-universal named HTML entities, or convert to numeric ones; see section:- #3.2

'0' - convert '1' - allow *

no_deprecated_attr Allow deprecated attributes or transform them; see section:- #3.4.6

'0' - allow
'1' - transform, but 'name' attributes for 'a' and 'map' are retained * '2' - transform

parent Name of the parent element, possibly imagined, that will hold the input; see section:- #3.3

safe Magic parameter to make input the most secure against vulnerabilities like XSS without needing to specify other relevant '$config' parameters; see section:- #3.6

'0' - no * '1' - will auto-adjust other relevant '$config' parameters (indicated by '"' in this list) ^

schemes Array of attribute-specific, comma-separated, lower-cased list of schemes (protocols) allowed in attributes accepting URLs (or '!' to deny any URL); '*' covers all unspecified attributes; see section:- #3.4.3

'href: aim, app, feed, file, ftp, gopher, http, https, javascript, irc, mailto, news, nntp, sftp, ssh, tel, telnet; :data, file, http, https, javascript' ^ 'href: aim, feed, file, ftp, gopher, http, https, irc, mailto, news, nntp, sftp, ssh, tel, telnet; style: !; *:file, http, https' "

show_setting Name of a PHP variable to assign the finalized '$config' and '$spec' values; see section:- #3.8

style_pass Ignore 'style' attribute values, letting them through without any alteration

'0' - no * '1' - htmLawed will let through any 'style' value; see section:- #3.4.8

tidy Beautify or compact HTML code; see section:- #3.3.5

'-1' - compact '0' - no * '1' or string - beautify (custom format specified by 'string')

unique_ids 'id' attribute value checks; see section:- #3.4.2

'0' - no
'1' - remove duplicate and/or invalid ones * word - remove invalid ones and replace duplicate ones with new and unique ones based on the word; the admin-specified word cannot contain a space character

valid_xhtml Magic parameter to make input the most valid XHTML without needing to specify other relevant '$config' parameters; see section:- #3.5

'0' - no * '1' - will auto-adjust other relevant '$config' parameters (indicated by '~' in this list)

xml:lang Auto-add 'xml:lang' attribute; see section:- #3.4.1

'0' - no * '1' - add if 'lang' attribute is present '2' - add if 'lang' attribute is present, and remove 'lang' ~

-- 2.3 Extra HTML specifications using the $spec parameter --------o

The '$spec' argument of htmLawed can be used to disallow an otherwise legal attribute for an element, or to restrict the attribute's values. This can also be helpful as a security measure (e.g., in certain versions of browsers, certain values can cause buffer overflows and denial of service attacks), or in enforcing admin policies. '$spec' is specified as a string of text containing one or more rules, with multiple rules separated from each other by a semi-colon (';'). E.g.,

$spec = 'i=-*; td, tr=style, id, -*; a=id(match="/[a-z][a-z\d.:\-`"]*/i"/minval=2), href(maxlen=100/minlen=34); img=-width,-alt';
$processed = htmLawed($text, $config, $spec);

Or,

$processed = htmLawed($text, $config, 'i=-*; td, tr=style, id, -*; a=id(match="/[a-z][a-z\d.:\-`"]*/i"/minval=2), href(maxlen=100/minlen=34); img=-width,-alt');

A rule begins with an HTML element name(s) (rule-element), for which the rule applies, followed by an equal-to (=) sign. A rule-element may represent multiple elements if comma (,)-separated element names are used. E.g., 'th,td,tr='.

Rest of the rule consists of comma-separated HTML attribute names, which can be the wildcard references '', 'aria', 'data', and 'on' for the sets of all standard, Aria, data-, and event (on) attributes, respectively. A minus (-) character before an attribute means that the attribute is not permitted inside the rule-element. E.g., '-width'. To deny all attributes, '-' can be used. All Aria, data-, and event (on) attributes can similarly be denined using 'aria', 'data', and 'on', respectively.

Following shows examples of rule excerpts with rule-element 'a' and the attributes that are being permitted:

• 'a=' - all
• 'a=id' - all
• 'a=href, title, -id, -onclick' - all except 'id' and 'onclick'
• 'a=*, id, -id' - all except 'id'
• 'a=-*' - none
• 'a=-*, href, title' - none except 'href' and 'title'
• 'a=-*, -id, href, title' - none except 'href' and 'title'

• 'a=-on, -id, href, onclick, title' - all except 'id' and on other than 'onclick'

  Rules regarding attribute values are optionally specified inside round brackets after attribute names – which cannot be wildcard references like '' or 'data' – in solidus (/)-separated parameter = value pairs. E.g., 'title(maxlen=30/minlen=5)'. None or one or more of the following parameters may be specified:

• 'oneof' - one or more choices separated by '|' that the value should match; if only one choice is provided, then the value must match that choice; matching is case-sensitive

• 'noneof' - one or more choices separated by '|' that the value should not match; matching is case-sensitive

• 'maxlen' and 'minlen' - upper and lower limits for the number of characters in the attribute value; specified in numbers

• 'maxval' and 'minval' - upper and lower limits for the numerical value specified in the attribute value; specified in numbers

• 'match' and 'nomatch' - pattern that the attribute value should or should not match; specified as PHP/PCRE-compatible regular expressions with delimiters and possibly modifiers (e.g., to specify case-sensitivity for matching)

• 'default' - a value to force on the attribute if the value provided by the writer does not fit any of the specified parameters

  If 'default' is not set and the attribute value does not satisfy any of the specified parameters, then the attribute is removed. The 'default' value can also be used to force all attribute declarations to take the same value (by getting the values declared illegal by setting, e.g., 'maxlen' to '-1').

  Examples with input '' are shown below.

  Rule: 'input=title(maxlen=60/minlen=6), value' Output: ''

  Rule: 'input=title(), value(maxval=8/default=6)' Output: ''

  Rule: 'input=title(nomatch=%w.d%i), value(match=%em%/default=6em)' Output: ''

  Rule: 'input=class(noneof=ic2|ic3/oneof=ic1|ic4), title(oneof=height|depth/default=depth), value(noneof=5|6)' Output: ''

  Special characters: The characters ';', ',', '/', '(', ')', '|', '~' and space have special meanings in the rules. Words in the rules that use such characters, or the characters themselves, should be escaped by enclosing in pairs of double-quotes ('"'). A back-tick ('') can be used to escape a literal '"'. An example rule illustrating this is 'input=value(maxlen=30/match="/^\w/"/default="your"ID`"")'.

  Attributes that accept multiple values: If an attribute is 'accesskey', 'class', 'itemtype' or 'rel', or 'archive' in case of 'object' element, which can have multiple, space-separated values, or 'archive' in case of 'object' element and 'srcset', which can have multiple, comma-separated values, htmLawed will parse the attribute value for such multiple values and will individually test each of them. The parsing is performed after any URL assessment of the attribute values (section:- #3.4.3).

  Note: To deny an attribute for all elements for which it is legal, '$config["deny_attribute"]' (see section:- #3.4) can be used instead of '$spec'. Also, attributes can be allowed element-specifically through '$spec' while being denied globally through '$config["deny_attribute"]'. The 'hook_tag' parameter (section:- #3.4.9) can also be possibly used to implement a functionality like that achieved using '$spec' functionality.

  Note: Attributes permitted through '$spec' are permitted regardless of any denial through '$config'. An attribute for which $spec indicates both permission and denial will be permitted. E.g., 'onclick' with '$spec' value of 'a = , -onclick, onclick', 'a = -on, onclick' or 'a = on*, -onclick' will be permitted inside 'a'.

  Note: Attributes' specifications for an element may be (inadvertently) set through multiple rules. In case of conflict, the attribute specification in the first rule will get precedence.

  '$spec' can also be used to permit custom or non-standard attributes. Thus, the following value of '$spec' will permit the custom uses of the standard 'rel' attribute in 'input' (not permitted as per standards) and of a non-standard attribute, 'vFlag', in 'img'.

  $spec = 'img=vFlag; input=rel'

  The attribute names must begin with an alphabet and cannot have space, equal-to (=) and solidus (/) characters.

-- 2.4 Performance time & memory usage ----------------------------o

The time and memory consumed during text processing by htmLawed depends on its configuration, the size of the input, and the amount, nestedness and well-formedness of the HTML markup within the input. In particular, tag balancing and beautification each can increase the processing time by about a quarter.

The htmLawed demo:- htmLawedTest.php can be used to evaluate the performance and effects of different types of input and '$config'.

-- 2.5 Some security risks to keep in mind ------------------------o

When setting the parameters/arguments (like those to allow certain HTML elements) for use with htmLawed, one should bear in mind that the setting may let through potentially dangerous HTML code which is meant to steal user-data, deface a website, render a page non-functional, etc. Unless end-users, either people or software, supplying the content are completely trusted, security issues arising from the degree of HTML usage permitted through htmLawed's setting should be considered. For example, following increase security risks:

• Allowing 'script', 'applet', 'embed', 'iframe', 'canvas', 'audio', 'video', 'dialog' or 'object' elements, or certain of their attributes like 'allowscriptaccess'

• Allowing HTML comments (some Internet Explorer versions are vulnerable with, e.g., ''

• Allowing dynamic CSS expressions (some Internet Explorer versions are vulnerable)

• Allowing the 'style' attribute

  To remove unsecure HTML, code-developers using htmLawed must set '$config' appropriately. E.g., '$config["elements"] = "* -script"' to deny the 'script' element (section:- #3.3), '$config["safe"] = 1' to auto-configure ceratin htmLawed parameters for maximizing security (section:- #3.6), etc.

  Permitting the 'style' attribute brings in risks of click-jacking, phishing, web-page overlays, etc., even when the 'safe' parameter is enabled (see section:- #3.6). Except for URLs and a few other things like CSS dynamic expressions, htmLawed currently does not check every CSS style property. It does provide ways for the code-developer implementing htmLawed to do such checks through htmLawed's '$spec' argument, and through the 'hook_tag' parameter (see section:- #3.4.8 for more). Disallowing 'style' completely and relying on CSS classes and stylesheet files is recommended.

  htmLawed does not check or correct the character encoding of the input it receives. In conjunction with permissive circumstances, such as when the character encoding is left undefined through HTTP headers or HTML 'meta' tags, this can allow for an exploit (like Google's UTF-7/XSS vulnerability of the past).

  Ocassionally, though very rarely, the default settings with which htmLawed runs may change between different versions of htmLawed. Admins should keep this in mind when upgrading htmLawed. Important changes in htmLawed's default behavior in new releases of the software are noted in section:- #4.5 on upgrades.

-- 2.6 Use with 'kses()' code -------------------------------------o

The 'Kses' PHP script for HTML filtering is used by many applications (like 'WordPress', as in year 2012). It is possible to have such applications use htmLawed instead, since it is compatible with code that calls the 'kses()' function declared in the 'Kses' file (usually named 'kses.php'). E.g., application code like this will continue to work after replacing 'Kses' with htmLawed:

$comment_filtered = kses($comment_input, array('a'=>array(), 'b'=>array(), 'i'=>array()));

If the application uses a 'Kses' file that has the 'kses()' function declared, then, to have the application use htmLawed instead of 'Kses', rename 'htmLawed.php' (to 'kses.php', e.g.) and replace the 'Kses' file (or just replace the code in the 'Kses' file with the htmLawed code). If the 'kses()' function in the 'Kses' file had been renamed by the application developer (e.g., in 'WordPress', it is named 'wp_kses()'), then appropriately rename the 'kses()' function in the htmLawed code. Then, add the following code (which was a part of htmLawed prior to version 1.2):

// kses compatibility
function kses($t, $h, $p=array('http', 'https', 'ftp', 'news', 'nntp', 'telnet', 'gopher', 'mailto')){
 foreach($h as $k=>$v){
  $h[$k]['n']['*'] = 1;
 }
 $C['cdata'] = $C['comment'] = $C['make_tag_strict'] = $C['no_deprecated_attr'] = $C['unique_ids'] = 0;
 $C['keep_bad'] = 1;
 $C['elements'] = count($h) ? strtolower(implode(',', array_keys($h))) : '-*';
 $C['hook'] = 'kses_hook';
 $C['schemes'] = '*:'. implode(',', $p);
 return htmLawed($t, $C, $h);
 }

function kses_hook($t, &$C, &$S){
 return $t;
}

If the 'Kses' file used by the application has been significantly altered by the application developers, then one may need a different approach. E.g., with 'WordPress' (as in the year 2012), it is best to copy the htmLawed code, along with the above-mentioned additions, to 'wp_includes/kses.php', rename the newly added function 'kses()' to 'wp_kses()', and delete the code for the original 'wp_kses()' function.

If the 'Kses' code has a non-empty hook function (e.g., 'wp_kses_hook()' in case of 'WordPress'), then the code for htmLawed's 'kses_hook()' function should be appropriately edited. However, the requirement of the hook function should be re-evaluated considering that htmLawed has extra capabilities. With 'WordPress', the hook function is an essential one. The following code is suggested for the htmLawed 'kses_hook()' in case of 'WordPress':

// kses compatibility
function kses_hook($string, &$cf, &$spec){   
 $allowed_html = $spec;
 $allowed_protocols = array();
 foreach($cf['schemes'] as $v){
  foreach($v as $k2=>$v2){
   if(!in_array($k2, $allowed_protocols)){
    $allowed_protocols[] = $k2;
   }
  }
 }
 return wp_kses_hook($string, $allowed_html, $allowed_protocols);
}

-- 2.7 Tolerance for ill-written HTML -----------------------------o

htmLawed can work with ill-written HTML code in the input. However, HTML that is too ill-written may not be read as HTML, and may therefore get identified as mere plain text. Following statements indicate the degree of looseness that htmLawed can work with, and can be provided in instructions to writers:

• Tags must be flanked by '<' and '>' with no '>' inside -- any needed '>' should be put in as '>'. It is possible for tag content (element name and attributes) to be spread over many lines instead of being on one. A space may be present between the tag content and '>', like '

  ' and '<img / >', but not after the '<'.

• Element and attribute names need not be lower-cased.

• Attribute string of elements may be liberally spaced with tabs, line-breaks, etc.

• Attribute values may be single- and not double-quoted.

• Left-padding of numeric entities (like, ' ', '&x07ff;') with '0' is okay as long as the number of characters between between the '&' and the ';' does not exceed 8. All entities must end with ';' though.

• Named character entities must be properly cased. Thus, '≪' or '&TILDE;' will not be recognized as entities and will be neutralized.

• HTML comments should not be inside element tags (they can be between tags), and should begin with ''. Characters like '<', '>', and '&' may be allowed inside depending on '$config', but any '-->' inside should be put in as '-->'. Any '--' inside will be automatically converted to '-', and a space will be added before the '-->' comment-closing marker unless '$config["comments"]' is set to '4' (section:- #3.3.1).

• 'CDATA' sections should not be inside element tags, and can be in element content only if plain text is allowed for that element. They should begin with '<[CDATA[' and end with ']]>'. Characters like '<', '>', and '&' may be allowed inside depending on '$config', but any ']]>' inside should be put in as ']]>'.

• For attribute values, character entities '<', '>' and '&' should be used instead of characters '<' and '>', and '&' (when '&' is not part of a character entity). This applies even for Javascript code in values of attributes like 'onclick'.

• Characters '<', '>', '&' and '"' that are part of actual Javascript, etc., code in 'script' elements should be used as such and not be put in as entities like '>'. Otherwise, though the HTML will be valid, the code may fail to work. Further, if such characters have to be used, then they should be put inside 'CDATA' sections.

• Simple instructions like "an opening tag cannot be present between two closing tags" and "nested elements should be closed in the reverse order of how they were opened" can help authors write balanced HTML. If tags are imbalanced, htmLawed will try to balance them, but in the process, depending on '$config["keep_bad"]', some code/text may be lost.

• Input authors should be notified of admin-specified allowed elements, attributes, configuration values (like conversion of named entities to numeric ones), etc.

• With '$config["unique_ids"]' not '0' and the 'id' attribute being permitted, writers should carefully avoid using duplicate or invalid 'id' values as even though htmLawed will correct/remove the values, the final output may not be the one desired. E.g., when '' is processed into ''.

• Even if intended HTML is lost from an ill-written input, the processed output will be more secure and standard-compliant.

• For URLs, unless '$config["scheme"]' is appropriately set, writers should avoid using escape characters or entities in schemes. E.g., 'http' (which many browsers will read as the harmless 'http') may be considered bad by htmLawed.

• htmLawed will attempt to put plain text present directly inside 'blockquote', 'form', 'map' and 'noscript' elements (illegal as per the specifications) inside auto-generated 'div' elements during tag balancing (section:- #3.3.3).

-- 2.8 Limitations & work-arounds ---------------------------------o

htmLawed's main objective is to make the input text more standard-compliant, secure for readers, and free of HTML elements and attributes considered undesirable by the administrator. Some of its current limitations, regardless of this objective, are noted below along with possible work-arounds.

It should be borne in mind that no browser application is 100% standard-compliant, standard specifications continue to evolve, and many browsers accept commonly used non-standard HTML. Regarding security, note that unsafe HTML code is not legally invalid per se.

• htmLawed might not strictly adhere to current HTML standards as standard specification for HTML by WHATWG:- http://www.whatwg.org is continuously evolving, and there is laxity among HTML interpreters (browsers) regarding standards. Admins can configure htmLawed to be more strict about standard compliance.

• In general, htmLawed processes input to generate output that is most likely to be standard-compatible in most users' browsers. Thus, for example, it does not enforce the required value of '0' on 'border' attribute of 'img' (an HTML version 5 specification).

• htmLawed is meant for input that goes into the 'body' of HTML documents. HTML's head-level elements are not supported, nor are the frame-specific elements 'frameset', 'frame' and 'noframes'. However, content of the latter elements can be individually filtered through htmLawed.

• It cannot handle input that has non-HTML code like 'SVG' and 'MathML'. One way around is to break the input into pieces and passing only those without non-HTML code to htmLawed. Another is described in section:- #3.9. A third way may be to some how take advantage of the '$config["and_mark"]' parameter (see section:- #3.2).

• By default, htmLawed won't check many attribute values for standard compliance. E.g., 'width="20m"' with the dimension in non-standard 'm' is let through. Implementing universal and strict attribute value checks can make htmLawed slow and resource-intensive. Admins should look at the 'hook_tag' parameter (section:- #3.4.9) or '$spec' to enforce finer checks on attribute values.

• By default, htmLawed considers all ARIA, data-*, event, and microdata attributes as global attributes and permits them in all elements. This is not strictly standard-compliant. E.g., the 'itemtype' microdata attribute is permitted only in elements that also have the 'itemscope' attribute. Admins can configure htmLawed to be more strict about this (section:- #2.3).

• The attributes, whether deprecated (which can be transformed by htmLawed) or not, that it supports are largely those that are in the specifications. Only a few of the proprietary attributes are supported. However, '$spec' can be used to allow custom attributes (section:- #2.3).

• Except for contained URLs and dynamic expressions (also optional), htmLawed does not check CSS style property values. Admins should look at using the 'hook_tag' parameter (section:- #3.4.9) or '$spec' for finer checks. Perhaps the best option is to disallow 'style' but allow 'class' attributes with the right 'oneof' or 'match' values for 'class', and have the various class style properties in '.css' CSS stylesheet files.

• htmLawed does not parse emoticons, decode BBcode, or wikify, auto-converting text to proper HTML. Similarly, it won't convert line-breaks to 'br' elements. Such functions are beyond its purview. Admins should use other code to pre- or post-process the input for such purposes.

• htmLawed cannot be used to have links force-opened in new windows (by auto-adding appropriate 'target' and 'onclick' attributes to 'a'). Admins should look at Javascript-based DOM-modifying solutions for this. Admins may also be able to use a custom hook function to enforce such checks ('hook_tag' parameter; see section:- #3.4.9).

• Nesting-based checks are not possible. E.g., one cannot disallow 'p' elements specifically inside 'td' while permitting it elsewhere. Admins may be able to use a custom hook function to enforce such checks ('hook_tag' parameter; see section:- #3.4.9).

• Except for optionally converting absolute or relative URLs to the other type, htmLawed will not alter URLs (e.g., to change the value of query strings or to convert 'http' to 'https'. Having absolute URLs may be a standard-requirement, e.g., when HTML is embedded in email messages, whereas altering URLs for other purposes is beyond htmLawed's goals. Admins may be able to use a custom hook function to enforce such checks ('hook_tag' parameter; see section:- #3.4.9).

• Pairs of opening and closing tags that do not enclose any content (like '') are not removed. This may be against the standard specification for certain elements (e.g., 'table'). However, presence of such standard-incompliant code will not break the display or layout of content. Admins can also use simple regex-based code to filter out such code.

• htmLawed does not check for certain element orderings described in the standard specifications (e.g., in a 'table', 'tbody' is allowed before 'tfoot'). Admins may be able to use a custom hook function to enforce such checks ('hook_tag' parameter; see section:- #3.4.9).

• htmLawed does not check the number of nested elements. E.g., it will allow two 'caption' elements in a 'table' element, illegal as per standard specifications. Admins may be able to use a custom hook function to enforce such checks ('hook_tag' parameter; see section:- #3.4.9).

• There are multiple ways to interpret ill-written HTML. E.g., in 'text', is it that the second closing tag for 'small' is missing or is it that the second opening tag for 'small' was put in by mistake? htmLawed corrects the HTML in the string assuming the former, while the user may have intended the string for the latter. This is an issue that is impossible to address perfectly.

• htmLawed might convert certain entities to actual characters and remove backslashes and CSS comment-markers ('/*') in 'style' attribute values in order to detect malicious HTML like crafted, Internet Explorer browser-specific dynamic expressions like 'expression...'. If this is too harsh, admins can allow CSS expressions through htmLawed core but then use a custom function through the 'hook_tag' parameter (section:- #3.4.9) to more specifically identify CSS expressions in the 'style' attribute values. Also, using '$config["style_pass"]', it is possible to have htmLawed pass 'style' attribute values without even looking at them (section:- #3.4.8).

• htmLawed does not correct certain possible attribute-based security vulnerabilities (e.g., 'x'). These arise when browsers mis-identify markup in escaped text, defeating the very purpose of escaping text (a bad browser will read the given example as 'x').

• Because of inadequate Unicode support in PHP, htmLawed does not remove the high value HTML-invalid characters with multi-byte code-points. Such characters however are extremely unlikely to be in the input. (see section:- #3.1).

• htmLawed does not check or correct the character encoding of the input it receives. In conjunction with permitting circumstances such as when the character encoding is left undefined through HTTP headers or HTML 'meta' tags, this can permit an exploit (like Google's UTF-7/XSS vulnerability of the past). Also, htmLawed can mangle input text if it is not well-formed in terms of character encoding. Administrators can consider using code available elsewhere to check well-formedness of input text characters to correct any defect.

• htmLawed is expected to work with input texts in ASCII standard-compatible single-byte encodings such as national variants of ASCII (like ISO-646-DE/German of the ISO 646 standard), extended ASCII variants (like ISO 8859-10/Turkish of the ISO 8859/ISO Latin standard), ISO 8859-based Windows variants (like Windows 1252), EBCDIC, Shift JIS (Japanese), GB-Roman (Chinese), and KS-Roman (Korean). It should also properly handle texts with variable-byte encodings like UTF-7 (Unicode) and UTF-8 (Unicode). However, htmLawed may mangle input texts with double-byte encodings like UTF-16 (Unicode), JIS X 0208:1997 (Japanese) and K SX 1001:1992 (Korean), or the UTF-32 (Unicode) quadruple-byte encoding. If an input text has such an encoding, administrators can use PHP's iconv:- http://php.net/manual/en/book.iconv.php functions, or some other mean, to convert text to UTF-8 before passing it to htmLawed.

• Like any script using PHP's PCRE regex functions, PHP setup-specific low PCRE limit values can cause htmLawed to at least partially fail with very long input texts.

$config = array('safe'=>1);
$out = htmLawed($in, $config);

$out = htmLawed($in);

$config = array('schemes'=>'*:*; src:http, https');
$out = htmLawed($in, $config);

$config = array('safe'=>1, 'elements'=>'a, em, strong');
$out = htmLawed($in, $config);

$config = array('elements'=>'* -script -object');
$out = htmLawed($in, $config);

$config = array('deny_attribute'=>'id, style');
$out = htmLawed($in, $config);

$config = array('deny_attribute'=>'* -title -href');
$out = htmLawed($in, $config);

$config = array('keep_bad'=>0);
$out = htmLawed($in, $config);

$config = array('deny_attribute'=>'title, id, style, on*');
$spec = 'a=title';
$out = htmLawed($in, $config, $spec);

$spec = 'img=vFlag; input=rel';
$out = htmLawed($in, $config, $spec);

$processed = htmLawed($in, array('elements'=>'a, em, strike, strong, u', 'make_tag_strict'=>1, 'safe'=>1, 'schemes'=>'*:http, https'), 'a=href');

$processed = htmLawed($in, array('css_expression'=>1, 'keep_bad'=>1, 'make_tag_strict'=>1, 'schemes'=>'*:*', 'valid_xhtml'=>1));

$processed = htmLawed($in, array('elements'=>'tr, td', 'tidy'=>-1), 'tr, td =');

  <em>My</em> website is <a href="http://a.com>a.com</a>.

  <em>My</em> website is a.com.

  <em>My</em> website is &lt;a href=""&gt;a.com&lt;/a&gt;.

a, abbr, acronym, address, applet, area, article, aside, audio, b, bdi, bdo, big, blockquote, br, button, canvas, caption, center, cite, code, col, colgroup, command, data, datalist, dd, del, details, dfn, dialog, dir, div, dl, dt, em, embed, fieldset, figcaption, figure, font, footer, form, h1, h2, h3, h4, h5, h6, header, hgroup, hr, i, iframe, img, input, ins, isindex, kbd, keygen, label, legend, li, link, main, map, mark, menu, meta, meter, nav, noscript, object, ol, optgroup, option, output, p, param, picture, pre, progress, q, rb, rbc, rp, rt, rtc, ruby, s, samp, script, section, select, slot, small, source, span, strike, strong, style, sub, summary, sup, table, tbody, td, template, textarea, tfoot, th, thead, time, tr, track, tt, u, ul, var, video, wbr