yacs 7.11: AJAX, on-line chats, page templates, Freemind, security and performance
Posted by Bernard on Dec. 1 2007, commented by Bernard on Jan. 7, (popular)
AJAX has been fully integrated into YACS, including the prototype.js and scripta.culo.us.js famous libraries. These have been completed with our own Javascript library, named shared/yacs.js. Auto-completion has been integrated into several forms. For example, this is used to guide folksonomies while editing tags, and to select among users when assigning people to sections.
Page templates have been added to YACS to ease content management. You have to create one page first, and then to configure sections to use it as a template for every new article. Combine page templates with some WYSIWYG editor, such as TinyMCE, to achieve rich and structured dynamic sites.
YACS may be the CMS with the most comprehensive integration of Freemind maps. Not only can Freemind files be displayed on-line, but YACS can also generate mind maps dynamically out of section content.
Access restrictions are now fully cascaded throughout the content tree. This is a simple, yet powerful improvement, that will ease a lot the configuration of complex permission schemes.
As with any other web system, the burden put on the server relates to the richness of information computed and provided to the end user. To improve on this as much as possible a number of SQL statements have been profiled and optimized.
527 files have been modified or added since the previous release.
Here is a list of new or updated features coming with this release:
Ajax and javascript

A lot of Javascript code has been added to implement AJAX into YACS. The full
prototype and
scripta.culo.us libraries have been added to the
included/browser directory. And a new
shared/yacs.js library has been created for the YACS code executed on browser side.

To optimize the loading of the Javascript libraries a single script
included/browser/minify.php is now in charge of compiling
.js files of the
included/browser into one single cacheable web object. Another script
included/jscalendar/minify.php has been added to optimize the loading of the jsCalendar library as well.

The browser and AJAX-based part of YACS has been put in
shared/yacs.js. That's Javascript, not PHP.

A new
tools/ajax.php has been added to test and to demonstrate basic AJAX features in YACS.

The computation of surfer time zone is now fully delegated to some Javascript executed in browser window, and transmitted to YACS as cookie afterwards. Same approach has been implemented for Flash detection. See
shared/surfer.php
The Javascript used for Flash detection and loading has been changed from
included/browser/flashobject.js to
included/browser/swfobject.js
The inclusion of Flash objects now fully relies on the
SWFObject library, as a safe mean to detect and manage browser capability. See
codes/codes.php for some examples of this.

The
shared/safe.php has been extended to support JSON encoding and decoding from within YACS, to further support AJAX interactions.

The contact page for a user has been made fully AJAX, to better take into account the actual availibility of the target person. See
Surfer::build_user_contact() in
shared/surfer.php
Many javascript pieces are now bracketed with XML-compliant tags, instead of the previous HTML comments. See
help.php for a live example of this.

The Javascript file used to support smileys has been made XHTML compliant. See
smileys/edit.js
Code that used to be part of
tables/sort.js has been moved to
shared/yacs.js and optimized

Auto-completion on user names has been implemented on server-side in
users/complete.php and in
users/layout_users_as_complete.php. The function
Users::get_options() used previously to build drop-down lists of users has been removed from
users/users.php
To better support AJAX updates YACS now provides
services/check.php as an easy way to detect page changes remotely.

A new script
categories/complete.php has been introduced to support tags auto-completion. This is the one that is called by
shared/yacs.js through AJAX during form handling.

A new alerting system has been put in place, to use interactive popups instead of mail messages when users are connected to the site.
Interactive pages with chat

All articles of a section can now appear as dynamic chats, when the option
view_as_thread is added in
sections/edit.php
The regular rendering script for articles can now be supplemented by using the option pattern
'views_as_xxx'. For example, use
'view_as_thread' to transform an article to an interactive chat place. This option loads the script
articles/view_as_thread.php.

A new layout
comments/layout_comments_as_thread.php has been added to shape live contributions to interactive threads.

The AJAX back-end scripts used for thread updates are
comments/thread.php
YACS now allows for one-to-one private messages, or chats, and the AJAX back-end for this has been implemented in
users/contact.php
A new parameter has been added to the configuration panel for users (in
users/configure.php) to allow for real-time alerts when surfers are connected to the server. These replace email messages sent by default. The actual dispatch is implemented in
Users::alert() in
users/users.phpPage templates and global pages

A new field has been added to sections to list one or several named pages as templates. This can be modified in advanced options from the web form used to edit a section (see
sections/edit.php). The template will be duplicated (including the overlay, if any) to create new pages in the section.

The page editor for articles now supports page templates. When a new article is created in a section that has been configured to use a page template, the template is duplicated before filling the capture form. A section can even support several page templates, in which case the editor will ask which one to use before moving forward. See
articles/edit.php
A new
select overlay has been created, that is only aiming to select another overlay. It is used in the section for template pages, to allow for the easy construction of model pages that include any kind of overlay. See
overlays/select.php
Previously, YACS was able to manage several cover pages in a dedicated section, and only the newest published article was actually used as a cover page. This has been streamlined, and now you have one single article named 'cover' to do the same job. At least the cover page can be loaded with one single SQL request instead of two.

Some pages which were implemented as dedicated scripts have been turned to regular named articles. For example, the
about.php has been replaced by an article with the nick name "about".
privacy.php has been changed to an article as well. See
articles/populate.php
All global pages, such as "about", "privacy", "cover" and "menu" have been grouped into one single section featured at the site map. Of course, "cover" and "menu" are visible only to associates.
Freemind

The content tree can now be displayed at site front page as an interactive Freemind object. This is one option of the new configuration panel for the front page, and other layouts can be selected as well, including customized PHP classes implementing the layout interface. See
index.php
Index pages of sections can now be rendered as an interactive Freemind map built dynamically by YACS based upon section content. Actually, this can be configured as a new layout option for sub-sections, in
sections/edit.php. The actual computation of the Freemind map is made in
articles/layout_articles_as_freemind.php
Any section index page can now be configured to appear as Freemind map (see
sections/edit.php). In this case YACS builds a map out of dynamic content (see
sections/layout_sections_as_freemind.php) and displays it in the middle of the index page. This may use the browsing of complex content trees.

An updated version of the Freemind player has been integrated. See
included/browser/visorFreemind.swf
A new script has been added to package the full content of one section in a dynamic Freemind map, in
sections/freemind.php
New codes have been added to integrate Freemind maps into pages. You can ask YACS to create a map dynamically, based on actual content of one section, or reference a Freemind static file attached to a page. In both cases YACS will include an interactive Freemind player (Flash or Java) to allow for interactive browsing of the map. See
codes/codes.php and
codes/live.php.
Enhanced security

The cascading mechanism of access permissions has been streamlined in
shared/anchors.php. The code used to load an anchor instance has been moved to this script from
shared/global.php.

The Anchor interface has been extended to better support the new cascading scheme for access permissions, and to allow for easy AJAX support. See
shared/anchor.php
Permission mask used for files and for directories is now displayed in the system test script, at
control/test.php
The permission system has been tested extensively, and a number of changes and improvements have been made to the software.

In case you experience some file permission issue you can trigger the
control/chmod.php script to have all file permissions set at once.

The list of users who have not authenticated for a long time, or who not contributed for a while, is now visible only to associates. See
users/review.phpPerformance

Pages generated by YACS now better conform to standards set by the Yahoo! performance team. Javascript are integrated at the end of pages, etc. See
shared/global.php
The rendering module for YACS codes has been improved with more efficient patterns, and positive results have been observed thanks to the new script dedicated to performance testing, at
codes/codes_test.php.

Code that replaces smileys by images has been entirely revisited to improve overall response times. Regular expressions used for string replacements have been profiled and optimized as well. See
smileys/smileys.php.

The loading of anchors has been transferred from
shared/global to
shared/anchors.php, and cache has been implemented there consistently

A new lightweight function has been introduced to beautify titles, with the aim to save on CPU cycles and memory during the construction of large and complex pages. Therefore
Codes::beautify_title() is now used instead of the regular
Codes::beautify() throughout most of the software. See
codes/codes.php.

YACS now integrates bare profiling support, to help identify performance issue, if any. See
shared/logger.php
Cache management has been entirely revisited, with scalability and performance in mind. YACS now allows for stable objects, that will be preserved during cache updates, except on full clearance. Also, only one SQL request is used to update the cache instead of two previously. See
shared/cache.php
Cached components have been consolidated in
articles/view.php to reduce the burden on the database.

This has been done in a number of other scripts as well, to further streamline the caching approach. There is no full-page cache anymore, because this creates per-user records in the cache, which definitely is not scalable for large communities. See for example
dates/month.php
All special calls to the MySQL library that were spread have been moved to
shared/sql.php to finalize the virtualization of the database engine.

Many SQL statements have been streamlined to reduce the burden on the database engine. For example, most SELECT using id rely now on numeric comparisons, instead of the LIKE operator used before.

Many statements using
LEFT JOIN syntax have been changed to a natural and straightforward joining approach, as in
comments/comments.php for example.

Also, the small SQL requests used to count items attached to articles and to sections have been streamlined to execute as fast as possible.

To allow for better performance YACS now supports the
mysqli extension if available, else it falls back to the regular
mysql extension.

The information panel for the database now displays if YACS uses the basic or the improved MySQL engine, and also if data are stored as Unicode or UTF-8 in tables. See
control/info.php.

YACS uses less strings than before for basic error messages, and the usage of these strings has been made more comprehensive throughout the software. For example, in
actions/actions.php, the message "No anchor for this action" has been turned to "No anchor has been found." Also, the names of parameter files have been extracted from strings in configuration panels, to limit the number of reference strings to a very minimum.

Cache requests to the database are now also cached to reduce the stress put on the back-end database server. See for example
Files::get() in
files/files.php
An
Expires: header has been added to web responses on stable objects such as the streaming files, etc. See
collections/play_audio.php for example.

YACS is also making a better usage of the HTTP protocol, by reflecting in status codes errors described in pages. This idea is to further help automated tools and search engines to detect and process abnormal conditions. See for example
categories/set_as_thumbnail.php.

YACS now always attempt to compress Javascript created dynamically, such as the outcome of
included/browser/minify.php, even if compression has not been explicitly allowed for other kinds of web objects. See
shared/global.php
Pre-computed of compressed Javascript files are provided in the archive file, just in case they could not be generated automatically at some site. Look in the
temporary directory of the archive file.

Cron is now desactivated when the server has been switched off. See
cron.php
Credentials sent by e-mail are now structured as JSON arrays to save on space. Of course, old credentials relying on PHP serialization are still accepted by
users/login.php
Most layouts for sections have been reviewed to streamline the code. See for example
sections/layout_sections_as_jive.php
The loading of the Google Maps API has been streamlined in
locations/locations.php
The table for links has been extended to speed SQL requests. See
links/links.php
The validation script now reports on memory consumption in
scripts/validate.php
Two new external scripts have been included in the standard YACS archive:
included/jsmin.php to compress Javascript files, and
included/json.php to help PHP scripts send Javascript arrays to the browser.
Other changes

Information stored in user profiles has grown dramatically, and it is now displayed progressively through a tabbed interface that loads content on-demand. The main tab show assigned sections and most recent posts, to get an immediate understanding of contributions from one user. Look the updated
users/view.php and the new AJAX back-end
users/element.php.

Real-time notifications are stored temporarily in the database and managed in
users/notifications.php
When YACS is configured to mask e-mail addresses, it also prevents direct mailing between users (in
users/mail.php). The goal is to avoid "internal spamming" from within public communities. One-to-one chat sessions are still allowed in this case, though.

The heartbit script has been entirely revisited to also allow for real-time notifications. On one hand it tracks user presence at some page. On the other hand it pushes back alert messages, if any. See
users/heartbit.php
Real-time presence at one page is now managed, in the back-end, by
users/visit.php. Transient records of user presence are saved in the database, abd managed through the library
users/visits.php
List of users now distinguishes between active and idle users. A different CSS class is given to idle users to allow proper rendering through cascaded style sheet. See
users/layout_users.php
YACS now supports Google Analytics. Add the Urchin account to use in the configuration panel for web services, in
services/configure.php and related Javascript will be generated automatically.

The configuration panel for skins now has many parameters related to Freemind visual rendering. Also, an important part of the code has been moved to the configuration panel for the home page, which is now separate. See
skins/configure.php and
configure.php
YACS now uses
digital as the default skin on any new installation.

Reference skins have changed, partly because of the introduction of global pages ('about', 'privacy'), and partly to optimize response times. See
skins/joi/template.php as an example.

A new
skins/_reference directory has been added to store reference files used by YACS in skins. The purpose is to simplify the integration of new styles and visual elements in existing skins. Now, you only have to add a directive
@import url("../_reference/yacs.css"); at the bery beginning of an existing style sheet to benefit from future evolutions.

The nicetitle style rules have been moved to
skins/_reference/yacs.css, so there is no need to have a separate directive to load
nicetitle.css anymore.

On the other hand, the nicetitle Javascript, which used to be systematically loaded by YACS, is now an option of each skin. To still benefit from the nicetitle effect you will have to copy
nicetitle.js to the directory of your skin, and to add loading directive to
template.php.

E-mail address obfuscation has been removed in
skins/skin_skeleton.php
All calls of
split() have been changed to
explode() instead.

Links generated by icons in layouts for the home page now support URL rewriting. See
skins/layout_home_articles_as_daily.php for example.

Titles of PDF files generated by YACS now support accented characters and UTF-8 input. See
shared/pdf.php
The form to change a user profile has been entirely re-structured. It now features several panels, to ease the job. See
users/edit.php
The page used to link users to a page or to a section now features auto-completion, to ease assignments. See
users/select.php
Directories
skins/images/avatars and
users/authenticators have been added to
tools/build_i18n.bat, the script that extracts strings from YACS source code to prepare for translation.

Tags have been added to user profiles (in
users/edit.php) for easy categorization. Also, categories now list jointly tagged users and pages, where applicable. See
categories/view.php. The purpose is to have a straightforward approach to link people with content, through categories.

User workstations can now be reached through an external network address set in user profiles (in
users/edit.php). This is used by YACS to drive people to one desktop shared in VNC during an on-line discussion, despite firewalls or reverse proxies. See
articles/view_as_thread.php
Watch lists have been extended to users, meaning you can monitor what others are doing. Visit any profile of interest and click on the Watch link to expand your social network (in
users/view.php and
users/track.php). Also, the list of watched people is built in
users/layout_users_as_watch.php
Several libraries are now loaded in
shared/global.php, reducing the need for many
include_once directive in all scripts. Code to handle values, visits, articles, sections, smileys is now available by default.

When
$HTTP_RAW_POST_DATA is not available, YACS reads
php://input instead. See for example
services/xml_rpc.php
Unicode entities are now properly transformed to UTF-8 in RSS feeds. See
services/rss_codec.php
Also, a new encoding function has been introduced to ensure XHTML conformance of links, for example in RSS feeds. See
services/rss_codec.php
The Content Assistant has been extended to create new sample sections, namely: 'global' (for global pages), 'channels' (for interactive chats), 'template' (for model pages), plus 'queries' and 'processed_queries'. Moreover, all queries feature a link to move them to the 'processed_queries' sections, thanks to the new behavior
move_on_article_access. See
sections/populate.php
The Yahoo! layout for sections now lists the proper number of sections, in
sections/layout_sections_as_yahoo.php
The TinyMCE library has been modified to make it XHTML compliant. Look at
include/tiny_mce/tiny_mce_src.js
The script used to create a new yacs archive now properly reports the name of the generated file. See
scripts/build.php
The form that initiates a yacs update now reflects the order of preferred methods, starting with the upload of the full archive. See
scripts/stage.php
Two new tables have been added to the database, to manage real-time visits, and to store interactive notifications. See
users/notifications.php and
users/visits.

On cache purge, yacs now also deletes files in the temporary directory. See
control/purge.php
An updated version of
control/import_csv.php has been provided by Tof and integrated into this release.

The Control Panel now features a tabbed interface to better structure all commands provided to site associates. See
control/index.php
Staged scripts are now
chmod to avoid some tricky issues related to bad file permissions in
scripts/update.php
A number of scripts to be executed only once have been introduced, to further help migrate sites running a previous version of yacs.

The deletion of a section with some content is now authorized, to ease the management of big sites. See
sections/delete.php
Tags have been added to user profiles, and that means that category pages are able to list people and content related to the same keyword. See
categories/view.php.

XHTML compliance has been greatly improved through a number of small modifications. For example, many forms have been reviewed to insert an additional
<div> tags, such as
categories/select.php.

Flash files (with the extension
.swf) can now be delivered natively across the wire, meaning a full-page display in the browser window. See
files/stream.php
The export to MS-Word now generates Unicode text, since UTF-8 is not supported natively by the tool from Microsoft. Same change has been applied to the PDF export. See
articles/fetch_as_msword.php and
articles/fetch_as_pdf.php
Layouts for articles have been modified to support live titles, that is, titles that can reflect internal state, as managed within overlays. For example, with the overlay
issue, titles of closed cases feature a nice
[Closed] suffix to make this explicit.

Several new layouts have been introduced for articles, including:
layout_articles_as_freemind.php,
layout_articles_as_select.php,
layout_articles_as_simple.php, and
layout_articles_as_threads.php.

The mail function used to process e-mail recipients. Now it can also parse nick names, if any, meaning you just have to remember nick names to invite people to visit one of your page. Moreover, the invitation form supports AJAX auto-completion to help identify target members. See
articles/mail.php
To streamline SEO efforts, YACS now automatically adds titles to page links. Previously this was based on nick names, and page authors had to configure nick names to make it work.

New code has been add to streamline the deletion of compound objects in articles, sections, cateories, user profiles, etc. Now a full section, and all of its content, can be deleted in two clicks.

The installation of SecureID adds a suffix to every web requests that prevents YACS to properly detect language in the browser. This has been fixed in
i18n/i18n.php.

Parameters can now be transmitted to layouts used for articles. For example, you can configure a section to layout articles like this: 'simple no_anchor'. This will trigger the
articles/layout_articles_as_simple.php script as usual, but also this will instruct the script to not insert links to page anchors.

When a file is attached to a page viewed as a dynamic thread, the list of files is updated, and the attachment is also referenced in the chatting area. See
files/thread.php
The file
readme.txt placed in the YACS archive has been updated.

Several flavors of
.htaccess are provided to easily deal with common situations.

User presence is now tracked and his record is updated in the database on each page visited. See
articles/view.php
Random names have been added to the script that populate comments in sample discussion threads, in
comments/populate.php.

The backup script now better leverages the intrinsic power of MySQL to reflect the database structure. See
control/backup.php
Two new parameters have been added to the system configuration panel to adjust permission masks used by YACS to manage files and directories, in
control/configure.php
The script
included/tar.php has been modified to accomodate for YACS permission masks.

The behavior interface can now add links to pages. This can be useful to implement bare workflows. For example, to move user queries to another section when an answer has been provided to the initial question. See
behaviors/behavior.php and the new behavior
behaviors/move_on_article_access.php
The
index.php now loads a different skin variant when it is invoked as the topmost page ('slash') and when it is called as regular YACS home page ('home'). This to give more flexibility to web designers, of course.

A new configuration panel dedicated to the front page has been added, in
configure.php. Moreover, associates have a Configure command directly at the front page itself, to access it in one click. Alternatively, the new configuration panel is also listed in the Control Panel.

The configuration panel for the front page also allows for customized layout for recent articles.