I’m posting this WordPress security message because I’ve made a change to CSV 2 POST security that can cause WordPress to die and display “Are you sure you want to do this?”.

Nonces

The security changes are regarding nonces. A nonce is created for every form and checked for every submission. However there may still be a couple of forms that work differently and if I miss them. It may trigger “Are you sure you want to do this?” to be displayed. I’m posting this to help remind us of these recent changes.

Function csv2post_include_form_processing_php() calls check_admin_referer(). Start here if the vague WordPress message is ever displayed while using CSV 2 POST. The function is in wtgcore_wp_config.php.

If you do a general search on the “Fatal error: Maximum execution time of 30 seconds exceeded ” error you will find it is a very common event. It may cause a site to come down for 2-3 minutes. Ten minutes downtime is possible if your a frequent offender. The fatal error is the server taking control of the situation and the limit, which is often 30 seconds. Is in place for both security and to reduce bandwidth usage on shared servers, something experienced on low cost hosting.

The cause is either a fault where a loop never ends and is meant to or in the case of a plugin like CSV 2 POST it is caused when the user requests the plugin to create more posts than their server can handle in one go. That is why we have the ability to break it down and create maybe 1000 posts at a time.

We are always working to improve that number. The most simple script can create 1 million posts in 30 seconds but they are basic posts on the most basic blogs.

Reduce Processing Time

There is a not lot of scope for using functions just for the sake of it but there is a gradual use of switches in the plugin that active functions i.e. once switched on certain functions will come into play during post creation. An example is text spinning, which most users will not use and so we first need to submit a form to switch text spinning on. This then puts all post content through text spinning functions.

Some things you can do to reduce processing time per post…

• Do not use functions in CSV 2 POST unless you really need them i.e. do not switch them on in Your Project configuration unless you intend to use the related feature or just remember to switch them back off.
• Process media first i.e. if you need your images to be cropped to various sizes including for the use as feature images you can upload them all in a batch fashion and have the images ready to be attached to new posts.
• Disabled plugins that react to anything being created should be disabled where possible. Twitter plugins are a big no unless automatic tweeting is off. Even if a schedule system is available for a tweets that is still extra work for WordPress during the creation of hundreds of posts.
• Switch pinging off and any other similiar services that update external domains.

Categories First Posts Second

As of 15th February 2013 CSV 2 POST makes users create their categories prior to posts being created to help reduce processing time. We may be able to take this approach with other elements of a post, one step at a time though.

Insight Into WordPress Post Creation

The average user won’t know what happens per post in terms of server processing and database queries. Some understanding into this helps to manage a plugin like CSV 2 POST which lets face it, uses WordPress in a way it was not designed for. Developers of WordPress do not put protection in place for a “Maximum execution time of 30 seconds”.

WordPress In Normal Use

WordPress was designed for manually creating posts and there is no way of getting around the fact that a single post requires multiple database queries. Plus with a lot of plugins installed we may increase processing time when extra things are done in reaction to a post being published i.e. a Twitter plugin may send a tweet or schedule one to be tweeted. Here are some of the queries WordPress might perform when manually creating a post…

• Permalink is established using Ajax straight away but it does require two queries.
• Tags are also stored prior to the Edit Post screen being submitted and post being published. But again more queries are done.
• Featured image requires a locally stored image, usually cropped to fit the theme and a query is made to the database to attach the media to the post.
• Media itself is stored in the database and that may require many queries
• Custom fields on the Edit Post screen are processed using Ajax so once again there is work for WordPress to do and it gets done as we go along.
• Now we submit our post and a query is done to store content.
• If we used categories a query is done to update a categories post counter, I think. It has been a while since I looked at this but it is a good example of how the smallest things require some communication with the database.

Auto-Blogging Plugins

Auto-blogging can trigger this error very easily, especially on slower servers. The slower a server, the longer it takes to create just a few hundred posts and it uses up extra seconds.  A great mass posting plugin like CSV 2 POST with all its advanced features takes even longer, if you use those features that is.

It is important to understand that CSV 2 POST uses WordPress core functions to create posts. All values are processed through WordPress functions so that plugins can get their chance to do what they are meant to do during post creation. The main reasons however are for the sake of standards and security.

Here is the above list of processes again but from an auto-blog perspective…

• Permalink is established at the same time as everything else. Meaning WordPress needs to query existing permalinks and establish a unique one before allowing the post to be stored in the database. Keep in mind, a draft is also created, there is no way around that because we need to secure our post ID and permalink. Permalink features in CSV 2 POST may add further processing.
• Tags don’t cause a query to be performed, we simple built the string of tags. However use of all settings will increase processing time per post especially if we automatically extract tags from snippets of text content.
• Featured images and any media processing may involve new files being created, queries to media related data and there is a good chance that other plugins or themes will be included in the process.
• Custom fields offer support for many advanced features in CSV 2 POST. We need to store various data to track a posts history and more. We can easily add an extra 10 custom fields per post if we use the full scope of this plugins abilities. The abilities are great and I wouldn’t avoid using them for the sake of processing time but it is a fact that each custom field requires a new record in the database.

I’ve added the ability to display all errors by adding a value to the URL on the admin side only. This will come version 6.8.3 within the next few days.

Example URL

http://www.csv2post.com/wp-admin/admin.php?page=csv2post_yourprojects&csv2postdebug=anyvalue

Using

Just add “csv2postdebug=something” at the end of your admin URL. You will notice I used “anyvalue” and “something” as it does not matter right now.

The Effect

WordPress hides what most users see as errors. Many of what people may think are errors are actual PHP notices, like warnings that indicate there could be a problem but not always. Sometimes they can be ignored. A lot of code in a lot of plugins and themes have such warnings so do not be alarmed if following the instructions above displays a lot of new lines on your screen.

It could look messy!

Simply remove the value from your URL and they will go away again. No one else can see the notices, just you. It does not change a setting it simply tells the plugin something like ‘During loading please show all errors’.

Why URL?

A setting used to active this ability has two issues. The first is that all users see the warnings so installing this plugin on busy admin and activating such a setting could be an issue. The second is that sometimes displaying notices triggers a problem that prevent the admin being loaded and so we cannot get back to the setting to switch it off. Doing it by URL is save.

However developers can do it by using at the top of csv2post.php

It must be rare that anyone uses a file name with spaces in it because for the first time a slight issue has come up. If we upload a .csv file with spaces in the files name it will break the interface.

Example

The image below shows the fault of a file with spaces. The file named “Full Profile_feed_2010 in order.csv” is the one sent to me by the first user to trigger this issue. We can see that the checkbox is not styled by jQuery UI like the other files in the list. Radio buttons in other columns are also not styled. This is not a major issue unless renaming the file is a problem.

Why?

The files name is used in various places in code. It is printed into HTML as form values, it is used in PHP for validation and arguments. The specific cause here is the use of the name in JavaScript where it is used in “id” attributes. The ID attribute does not accept spaces and so the script is effected for the item in the list of CSV files.

Functional Faults

I don’t know the full extent to what spaces will cause but I suspect it will cause other faults, not just interface related but functional. So please remove spaces.

Automatically Remove Spaces

If anyone needs such a function please request it. I could provide an option to activate it. I cannot do these things automatically without such a setting because some users may not want their file renamed. So a bit of work is required but I’m hoping that all users can rename the file without it being a problem for other operations in their project.