Neos 5.3 LTS and Flow 6.3 LTS Release

Long Term Support Version - LTS

This releases marks the first release to be supported until August 2023 - as well as all subsequent releases until Neos 6.2 / Flow 7.2.

Major changes since the last LTS

More than a year has passed since the last LTS Version. We focused a lot on improving things - in the code, but also in the language we are using, i.e. to make it more inclusive.
But to sum it up, these were major changes introduced since 4.3 (to name only a few):

• PSR-7 compatibility
• Many notable UI improvements, such as a fully responsive backend, inline property validation and improved error validation
• No more emberJS
• An all-new redirect handler backend module
• A lot of minor performance-improvements that sum up to a much better user experience 

All the details can be found in our former blog posts:

• Neos 5.0 »Jabberwock« and Flow 6.0
• Neos 5.1 »White Knight« and Flow 6.1
• Neos 5.2 »Bandersnatch« and Flow 6.2

Neos 5.3 LTS Highlights

Improved wording for include/exclude concepts

Words are not only "letters stringed together". They also include a lot of meaning. Within software development there are some include/exclude concepts based on the terminology of "white" which, to be super clear, has no correlation with something being "included"/"allowed" and "black" not with being "excluded"/"disallowed" under any possible point of view.

Therefore, some descriptions and methods have been renamed. To make the transition as easy as possible migration paths have been provided. This also applies for Flow 6.3 LTS.

• whitelist --> allowedMethods
• whitelisted --> allowed
• whitelistedObjects --> allowedObjects
• blacklisted --> excluded
• blacklistedSubPackages --> excludedSubPackages
• onlyWhitelistedObjects --> onlyAllowedObjects
• extensionBlacklist --> excludedExtensions
• and more

User Management for Users without Admin rights

This introduces a new role Neos.Neos:UserManager. If an editor has that role, they are able to manage users with roles not exceeding their own, without having administrator rights.

Made login controller view configurable using Views.yaml

The login controller view was not configurable using a Views.yaml (as was the case for Neos 3 & 4.) This is fixed by adding a Views.yaml and removing initializeView() from LoginController.

Now the Login screen can be customised again by creating custom Fusion for rendering it (see Neos.Neos/Resources/Private/Fusion/Backend for inspiration) and adjusting the path used through Views.yaml, e.g.:

Greatly speed up node move actions

Due to an issue in how Doctrine 2.x handles the computation of changesets (when given entities to commit) it recomputed the changesets for all entities for every entity. Leading to n^2 change computations.

In a large project this improved the moving of ~750 nodes from 1.7m to 8.5s. In the demo site moving the "features" page from 4s to 1.65s. Publishing seems to be only slightly (~10%) faster compared to its current behaviour in the CR.

What was done?

Now this behavior is circumvented by committing all entities at once. Including entities which might not have been included with the previous code but would have been persisted at the end of the request anyway.

What is breaking?

This change leads to all entities scheduled for persistence to actually persist when a node is changed, see following comparison timelines:

Old: 

1. Create Entity, mark for insertion
2. change node
3. controller call is done / persistAll was called -> entity from step one is now persisted

New:

1. Create Entity, mark for insertion
2. change node (entity will be persisted at this point)
3. controller call is done / persistAll was called -> nothing happens anymore.

How it was done

Replace the repeated flush calls to the entity manager with a single one.

This should only lead to a behavioural change if custom code would modify a node then move other nodes and expect the the first node was not persisted yet.

How to verify it

1. Move a large set of pages with subpages and nodes in the Neos backend.
2. Check the request time of the change xhr request.
3. Apply this patch
4. Repeat steps 1 + 2 and compare

Example screenshots:

Configurable Asset Constraints

Extends the Media Browser and Asset proxy search endpoint so that it supports constraints that filter the asset lists (on top of the user-specified filters).

This makes it possible to constraint asset source(s) and media type(s) per node type property. Example:

(restricts the asset editor to only allow audio files of the Neos or wikipedia_de asset source when using the media browser modal, search-box or file upload).

Added option to disable the creation of redirects for assets

This feature adds a config parameter for the media browser to disable the option to create redirects for replaced assets resources. Might be useful for some projects, where these kind of redirects don't make any sense and therefore should be disabled by default.

Added keyRenderer to render the key in the result map

This adds a new fusionPath keyRenderer to render the key of the resulting collection of Neos.Fusion:Map Example:

will render:

All Neos 5.3 LTS Changes

Highlights

• Improved wording of include/exclude concepts where possible
  • whitelist --> allowedMethods
  • whitelisted --> allowed
  • whitelistedObjects --> allowedObjects
  • blacklisted --> excluded
  • blacklistedSubPackages --> excludedSubPackages
  • onlyWhitelistedObjects --> onlyAllowedObjects
  • extensionBlacklist --> excludedExtensions
  • and many more
• BUGFIX: Speed up node move actions

Features

• User management for Users without Admin rights
• Made login controller view configurable using Views.yaml
• Configurable Asset Constraints
• Added option to disable the creation of redirects for assets
• Added keyRenderer to render the key in the result map

Improvements

• Removed deprecated code use from ContentCollectionRenderer
• Bumped lodash from 4.17.15 to 4.17.19 in /Neos.Neos
• Bumped websocket-extensions from 0.1.3 to 0.1.4 in /Neos.Media.Browser
• BUGFIX: Prevent refetching nodes for policy checks
  • When defining a policy targeting a static node the static node was refetched for every subject, slowing down policy information retrieval.
    This adds a simple caching mechanism to NodePrivilegeContext::getNodeByIdentifier that prevents nodes from being refetched for static policy comparisons.
• BUGFIX: Prevent refetching nodes for policy checks
  • When defining a policy targeting a static node the static node was refetched for every subject, slowing down policy information retrieval.
    This adds a simple caching mechanism to NodePrivilegeContext::getNodeByIdentifier that prevents nodes from being refetched for static policy comparisons.
• BUGFIX: Variants tab does not use mainRequest in form when opened from inspector
  • This adds a condition for using the parentRequest only if it is not the mainRequest already.
• BUGFIX: Make child nodes of hidden parents inaccessible
  • Since a couple of months child nodes of hidden nodes are accessible (outside the Neos backend). This change restores the initial behaviour and makes sure that accessing child nodes from hidden nodes will lead to a 404 response.
• BUGFIX: Fix login screen if no background image is set
• BUGFIX: pass copied node to emitAfterNodeCopy signal
• BUGFIX: Resolve error when backend session times out
• BUGFIX: Fix path to Styles/<Lite.css>
• BUGFIX: Add missing pagination styles for media browser
• BUGFIX: Passing glue string after array is deprecated
• BUGFIX: Respect fallback rule "strictness" in FusionView
  • When Neos finds a content dimension named "language" it uses that dimension to set the locale fallback order for rendering in the FusionView.
  • In Neos 5.0 that rule was switched to "strict" mode, meaning the order was used without falling back to implicit parents in locales. This broke translations in case the "language" dimension was configured with e.g. de_DE or en_US - for those cases translations were never used if the respective XLIFF files were in de (or en) folders.
  • This change makes the FusionView use the strict flag from the settings, giving back control to the user (in case non-strict is really needed). At the same time it makes translations work as would be expected in most cases, by using e.g. de_DE first, but falling back to de later.
• BUGFIX: Revert ContentCollection constraint change

Flow 6.3 LTS Highlights

Configurable PersistedUsernamePasswordProvider lookup name

Adds an option lookupProviderName to the PersistedUsernamePasswordProvider that allows the provider to be reused for multiple authentication types.

Example:

Browser arguments are the request parsed body

Since 6.0 the Browser $arguments were accidentally moved to the request query parameters, while they should reflect the post body arguments.

The docblock even states that: @param array $arguments Arguments to send in the request body

Note: this slightly changes behaviour if you adapted your usage of Browser since 6.0.0. If you want to send query parameters, you should instead provide an Uri instance with the parameters, as was already the case prior to 6.0. (This is the reason for marking this bugfix as breaking.)

Set session cookie after flash message storage

The FlashMessages component by default uses the SessionStorage, which depends on sending the session cookie through the SessionResponse component. However, the FlashMessages component was configured to be run after the session component. That way, unless a session was (still) open, the Set-Cookie was not sent and hence the flash message got lost. This was primarily visible after logging out and redirecting the user with a flash message.

Thanks @astehlik for finding the root cause!

Made PHP binary setting check use realpath

The check if the current executing php binary path as set in PHP_BINARY matches the given setting in phpBinaryPathAndFilename was case-sensitive. This check is only done when the setting is specified and should guarantee that the path specified refers to the same binary as the one executing the current script. On Windows, the filesystem is case-insensitive by default though (Windows 10 has options to make it case-sensitive) and the chance that someone accidentally gets the casing of the path wrong on windows is (arguably) much more likely than someone having different versions of php installed on a case-sensitive filesystem in paths /foo/PHP/php and /foo/php/php respectively.

So this will prevent irritating errors on Windows in the form of

Added configurable PSR-15 middleware kernel

This introduces a configurable middleware chain as the new main entry point into HTTP request handling. The old component chain is itself now a middleware "dispatch" that is configured to run at the end and other middlewares can be configured to run before, which also means it runs after "dispatch" due to how PSR-15 middlewares work.

Each middleware itself decides if it runs only before, after, or both by invoking the next handler appropriately. See also https://www.php-fig.org/psr/psr-15/meta/

This allows to easily manipulate the incoming HttpRequest or the outgoing HttpResponse with existing middlewares (e.g. https://github.com/middlewares/awesome-psr15-middlewares) and even completely short-cut Flows old component chain handling to handle special cases more performantly.

All Flow 6.3 LTS Changes

Features:

• Configurable PersistedUsernamePasswordProvider lookup name
• Browser arguments are the request parsed body
• Set session cookie after flash message storage
• Added configurable PSR-15 middleware kernel

Improvements:

• BUGFIX: Browser arguments are the request parsed body
• BUGFIX: Correct indention for throwables config
  • The configuration of the throwable exceptions should not be part of the PSR3 level. And the default config did not work since 6.0. The method initializeExceptionStorage still expects the configuration to be at the path Neos.Flow.log.throwables.
• Use argument unpacking instead of call_user_func_array
  • This is supported since PHP 5.6 and is actually faster than call_user_func_array at least since PHP 7. Just be sure to not use it in places where the method is static or a Closure. This also deprecates ObjectAccess::instantiateClass() (retrospectively). It will be removed with Flow 7.
• BUGFIX: Array.indexOf() returns an integer with associative arrays
• BUGFIX: Use serverRequestFactory to create server request with path
• BUGFIX: Check if PdoBackend cache tables exist in setup
  • Running the setup for a PdoBackend multiple times should not lead to an error, similar to other Backends.
• BUGFIX: Correct default log file size
• BUGFIX: Avoid trying to auto-wire constructor arguments for factory created objects
  • This bug was uncovered with the previous fix to not map arguments to the object when a factory is configured. That way (singleton) objects which expected a (non-optional/nullable) constructor argument that could not be auto-wired (e.g. because it was a scalar) would lead to the exception "Could not auto-wire required constructor argument ... for singleton class ...". This change fixes that, by completely skipping auto-wiring for objects configured with a factory. The factory has full responsibility to create and wire the constructor arguments correctly.
• BUGFIX: Allow Views to replace the response in AjaxWidgetContext
  • If a WidgetController has a view returning a PSR-7 response, it was not replaced in Ajax context. Now the behaviour of the ReplaceComponent is merged into the AjaxWidgetComponent, so the expected ActionController behaviour can be guaranteed
• BUGFIX: Renamed Doctrine\Common\Persistence -> Doctrine\Persistence
  • This fixes psalm errors due to the namespace renaming of various doctrine classes in doctrine/orm 2.7.3 release. The old class name was an alias since 2.7, so this should not break anything.
• BUGFIX: Allow objects to be serialised twice without loosing their relations to persisted properties

Upgrading

The detailed upgrading process for Neos 5.2 is explained in the upgrade instructions.

Changelogs

See the Neos 5.3 changelogs, Neos UI 5.3 changelogs and Flow 6.3 changelogs for all changes and details.

Support for Neos 5.3 and Flow 6.3

As shown in our release roadmap, Neos 5.3 and Flow 6.3 will get bug fixes until August 2022 and security fixes until August 2023.

Thanks to all core team members and the community for your valuable contributions and sponsorships. Especially during this difficult time in which a lot of people have many private topics on their mind and less time to contribute.

Take care everyone and stay healthy ❤️