[Tiki-devel] New Tiki Parser

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Tiki-devel] New Tiki Parser

Dr. Sassafras
Ive been thinking about the wiki parsing in tiki. Its a long-term goal of mine to make it faster, with more control over content.

Just to be clear. Ive got no immediate plans on coding this, its just a dream now. Was thinking that if we put our heads together, perhaps created a dev page, we could work out ahead of time what direction we want to see a new parser go in.


Wiki Editor

Pre-Save Processing

Database Copies Saved

Page viewed

Ajax Error checking reports Syntax errors, uninitiated plugins etc, before page is saved.

Wiki-Syntax is converted into XHTML5.

Source Page is Saved as XHTML5. (Wiki Processed) This page is saved even if it is not well formatted.

Any non-cached plugins are processed from the cached copy and the document is sent to client computer.

Page edits are always created from saved Source Pages. Edit request is processed to replace XHTML with Wiki Syntax. And sent to client for editing.

Formatting and Cached Plugins are evaluated.

Cached Copy Saved. Plugins rendered into XHTML5, with exception of non-cached plugins. This document is always well formatted.




Page index, created from evaluated cached copy. XHTML5 & Most whitespace removed.



Wiki Syntax – A set of rules that replaces shortcuts with XHTML5.

Multiple Definitions can be easy make applied and extended. eg. Tiki syntax, mediawiki syntax, or plain HTML5 can all be interchanged based on end-user preferences.


Formatting Plugins – Plugins that apply formatting, but do not modify any contents. These are swapped out for XHTML5, and are always cached. In comparison to wiki syntax, these plugins are always saved as a plugin, whereas the wiki syntax is fully interchangeable with XHTML5 structure.


Cached Plugin – A plugin that may modify the contents of what it encloses. These plugins are evaluated last before saving. Contents of these plugins must be wiki-parsed separately if that is desired. These plugs ins are saved in the cached page as XHTML5, and as a plugin within the saved page code.


Non Cached Plugin – Similar to the Cached plugin, but both the cached wiki page and wiki source page are saved as a plugin that is only evaluated upon viewing. These plugins will be in the minority, and can be used for plugin content that may change over time. (like including an external page)


Syntax processing can be handled with a regex (for Wiki-Syntax). Most processing there after can be processed with PHP built in DOM-XML functions.


Line Breaks (part of wiki syntax) should always evaluate into <br>’s never <p>’s. Its caused a lot of formatting issues in tiki, especially when evaluating sub-sections of text.


Most XHTML5 tags and attributes can be assigned as Approved, Denied or Approval Required. (whitelisted content) This allows minimal to no additional security filtering. Formatting plugin opening and closing tags are exempt from this, while there content is evaluated normally. (Perhaps this process could be assisted through DTD’s... perhaps multiple)


Plugins may also be marked as Approved, Denied, Approval Required. Approval Required may be assigned to a particular permission level, (Registered or Admin... or maybe group?) Denied formatting is saved in the cached copy as escaped XHTML5, while Pending Approval may be saved as the XHTML5 pending notice. When plugins-XHTML are approved, the auth-code can be saved as an attribute in the source page, and matched to a database entry.


Emoji and HTML special characters are treated as unicode, so no processing of these is required (with a few exceptions like reserved characters and nbsp etc.).




Brendan




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Tiki-devel] New Tiki Parser

Marc Laporte-3
Hi Brendan!

“You may say I'm a dreamer, but I'm not the only one.” ― John Lennon

https://dev.tiki.org/Wiki+Parser+Revamp

Thanks for bringing this up. Maybe something for Tiki19, just after Tiki18LTS?

M ;-)



On Tue, Apr 4, 2017 at 12:05 PM, Brendan Ferguson <[hidden email]> wrote:
Ive been thinking about the wiki parsing in tiki. Its a long-term goal of mine to make it faster, with more control over content.

Just to be clear. Ive got no immediate plans on coding this, its just a dream now. Was thinking that if we put our heads together, perhaps created a dev page, we could work out ahead of time what direction we want to see a new parser go in.


Wiki Editor

Pre-Save Processing

Database Copies Saved

Page viewed

Ajax Error checking reports Syntax errors, uninitiated plugins etc, before page is saved.

Wiki-Syntax is converted into XHTML5.

Source Page is Saved as XHTML5. (Wiki Processed) This page is saved even if it is not well formatted.

Any non-cached plugins are processed from the cached copy and the document is sent to client computer.

Page edits are always created from saved Source Pages. Edit request is processed to replace XHTML with Wiki Syntax. And sent to client for editing.

Formatting and Cached Plugins are evaluated.

Cached Copy Saved. Plugins rendered into XHTML5, with exception of non-cached plugins. This document is always well formatted.




Page index, created from evaluated cached copy. XHTML5 & Most whitespace removed.



Wiki Syntax – A set of rules that replaces shortcuts with XHTML5.

Multiple Definitions can be easy make applied and extended. eg. Tiki syntax, mediawiki syntax, or plain HTML5 can all be interchanged based on end-user preferences.


Formatting Plugins – Plugins that apply formatting, but do not modify any contents. These are swapped out for XHTML5, and are always cached. In comparison to wiki syntax, these plugins are always saved as a plugin, whereas the wiki syntax is fully interchangeable with XHTML5 structure.


Cached Plugin – A plugin that may modify the contents of what it encloses. These plugins are evaluated last before saving. Contents of these plugins must be wiki-parsed separately if that is desired. These plugs ins are saved in the cached page as XHTML5, and as a plugin within the saved page code.


Non Cached Plugin – Similar to the Cached plugin, but both the cached wiki page and wiki source page are saved as a plugin that is only evaluated upon viewing. These plugins will be in the minority, and can be used for plugin content that may change over time. (like including an external page)


Syntax processing can be handled with a regex (for Wiki-Syntax). Most processing there after can be processed with PHP built in DOM-XML functions.


Line Breaks (part of wiki syntax) should always evaluate into <br>’s never <p>’s. Its caused a lot of formatting issues in tiki, especially when evaluating sub-sections of text.


Most XHTML5 tags and attributes can be assigned as Approved, Denied or Approval Required. (whitelisted content) This allows minimal to no additional security filtering. Formatting plugin opening and closing tags are exempt from this, while there content is evaluated normally. (Perhaps this process could be assisted through DTD’s... perhaps multiple)


Plugins may also be marked as Approved, Denied, Approval Required. Approval Required may be assigned to a particular permission level, (Registered or Admin... or maybe group?) Denied formatting is saved in the cached copy as escaped XHTML5, while Pending Approval may be saved as the XHTML5 pending notice. When plugins-XHTML are approved, the auth-code can be saved as an attribute in the source page, and matched to a database entry.


Emoji and HTML special characters are treated as unicode, so no processing of these is required (with a few exceptions like reserved characters and nbsp etc.).




Brendan




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel




--

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Tiki-devel] New Tiki Parser

Dr. Sassafras
That would be amazing if it could be done so early. I would even be open to adding it as a second parser. There is some issues in the way that spacing is evaluated in wiki, and if we do a re-write we should probably set that right, so I’m presuming there might be small parts that are evaluated slight differently.  Although I’m sure 99% would be substantially identical, there might be a few things that we decide to break away from, so perhaps a assisted migration process might work. 

The prospect of that is very exciting. The inflexibility of the current parsing along with heavy system resources required (especially for very large complex wiki pages) is a current limiting factor for me. I would certainly be willing to be part of a team effort on the coding for this. 

Brendan



On Apr 4, 2017, at 5:14 PM, Marc Laporte <[hidden email]> wrote:

Hi Brendan!

“You may say I'm a dreamer, but I'm not the only one.” ― John Lennon

https://dev.tiki.org/Wiki+Parser+Revamp

Thanks for bringing this up. Maybe something for Tiki19, just after Tiki18LTS?

M ;-)



On Tue, Apr 4, 2017 at 12:05 PM, Brendan Ferguson <[hidden email]> wrote:
Ive been thinking about the wiki parsing in tiki. Its a long-term goal of mine to make it faster, with more control over content.

Just to be clear. Ive got no immediate plans on coding this, its just a dream now. Was thinking that if we put our heads together, perhaps created a dev page, we could work out ahead of time what direction we want to see a new parser go in.


Wiki Editor

Pre-Save Processing

Database Copies Saved

Page viewed

Ajax Error checking reports Syntax errors, uninitiated plugins etc, before page is saved.

Wiki-Syntax is converted into XHTML5.

Source Page is Saved as XHTML5. (Wiki Processed) This page is saved even if it is not well formatted.

Any non-cached plugins are processed from the cached copy and the document is sent to client computer.

Page edits are always created from saved Source Pages. Edit request is processed to replace XHTML with Wiki Syntax. And sent to client for editing.

Formatting and Cached Plugins are evaluated.

Cached Copy Saved. Plugins rendered into XHTML5, with exception of non-cached plugins. This document is always well formatted.




Page index, created from evaluated cached copy. XHTML5 & Most whitespace removed.



Wiki Syntax – A set of rules that replaces shortcuts with XHTML5.

Multiple Definitions can be easy make applied and extended. eg. Tiki syntax, mediawiki syntax, or plain HTML5 can all be interchanged based on end-user preferences.


Formatting Plugins – Plugins that apply formatting, but do not modify any contents. These are swapped out for XHTML5, and are always cached. In comparison to wiki syntax, these plugins are always saved as a plugin, whereas the wiki syntax is fully interchangeable with XHTML5 structure.


Cached Plugin – A plugin that may modify the contents of what it encloses. These plugins are evaluated last before saving. Contents of these plugins must be wiki-parsed separately if that is desired. These plugs ins are saved in the cached page as XHTML5, and as a plugin within the saved page code.


Non Cached Plugin – Similar to the Cached plugin, but both the cached wiki page and wiki source page are saved as a plugin that is only evaluated upon viewing. These plugins will be in the minority, and can be used for plugin content that may change over time. (like including an external page)


Syntax processing can be handled with a regex (for Wiki-Syntax). Most processing there after can be processed with PHP built in DOM-XML functions.


Line Breaks (part of wiki syntax) should always evaluate into <br>’s never <p>’s. Its caused a lot of formatting issues in tiki, especially when evaluating sub-sections of text.


Most XHTML5 tags and attributes can be assigned as Approved, Denied or Approval Required. (whitelisted content) This allows minimal to no additional security filtering. Formatting plugin opening and closing tags are exempt from this, while there content is evaluated normally. (Perhaps this process could be assisted through DTD’s... perhaps multiple)


Plugins may also be marked as Approved, Denied, Approval Required. Approval Required may be assigned to a particular permission level, (Registered or Admin... or maybe group?) Denied formatting is saved in the cached copy as escaped XHTML5, while Pending Approval may be saved as the XHTML5 pending notice. When plugins-XHTML are approved, the auth-code can be saved as an attribute in the source page, and matched to a database entry.


Emoji and HTML special characters are treated as unicode, so no processing of these is required (with a few exceptions like reserved characters and nbsp etc.).




Brendan




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel




--
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
TikiWiki-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
Loading...