Category Archives: Features

Spam is always a concern when you have email. We have successfully integrated DKIM signing to our email system. As it can break mail delivery this is not done automatically. DNS records need to be in place for it to work properly. Thus if you’re hosted at CLDMV please contact us in order to help you with the process of setting up DKIM.

Next upgrade we have put in place is our own custom mail filter. Right now there isn’t much to it but injecting a header into emails to show it working. But soon there will be SMTP Relay limits (used to prevent compromised emails from flooding emails outbound), Custom spam filtering though DB rules, and probably some other things which I’m not thinking of currently.

The two main issues we had with putting our own Filter in place was the ability to maintain our Anti-Virus and Spam systems that were already in place while also being able to provide a second content filter. After many hours and tons of caffeine we were able to produce a system which kept our current system intact while adding the secondary system on top of it. As always Security in this manner was a great concern. Especially considering we would be handling external content. Several limitations are at play in current mail systems which caused us to create innovate techniques in order to send emails to our own system securely. We even attempted to hack our own mail server through emails and were unsuccessful.

New feature for JS and CSS

June 12, 2014

Nathaniel Hyson

Features Information Servers Upgrades Why CLDMV No Comments

Any website hosted with CLDMV can now automatically serve up concatenated JS and CSS files. For example:

http://cdn.cldmv.net/cldmv/min/cldmv/jquery/?base64.js,decode.js

The above URL is two separate files put together and minified as well. Now lets go over the portions of the URL:

The prefix of the URL (this is required in order to get the script to work.


http://cdn.cldmv.net/cldmv/min/

http://cdn.cldmv.net/cldmv/min/

The directory:


cldmv/jquery/

cldmv/jquery/

Then the marker to indicate the files you want to serve (this is required and never changes):

?

Then finally the files you want to serve under the above directory:


base64.js,decode.js

base64.js,decode.js

Each file is separated by a comma.

Normally to get both of these scripts to run you would include the follow urls:


http://cdn.cldmv.net/cldmv/jquery/base64.js
http://cdn.cldmv.net/cldmv/jquery/decode.js

http://cdn.cldmv.net/cldmv/jquery/base64.js

http://cdn.cldmv.net/cldmv/jquery/decode.js

Instead you can now just include:


http://cdn.cldmv.net/cldmv/min/cldmv/jquery/?base64.js,decode.js

http://cdn.cldmv.net/cldmv/min/cldmv/jquery/?base64.js,decode.js

Update: Since Chrome has an issue with double question marks I’ve reduced the code to only require a single question mark.

Current Status of Control Panel, security updates, and server updates

June 2, 2014

Nathaniel Hyson

Development Downtime Features Information Information s1 Security Servers Upgrades Why CLDMV No Comments

First lets start with a screen shot of the dashboard in progress for the control panel.

Unfortunately Data prior to the 28th of May 2014 is a bit skewed. But as you can see we have plenty of navigation so consumers never get lost. We also show are currently showing all site traffic for all sites hosted under a specific user. Though with some crashes here and there we might put these charts on their own separate page per domain but still give the user the ability to view an all domain report as it’s setup right now (at their own risk).

Currently the theming for the charts is still to be completed. But we have the ability to show and hide specific points of data if we want, select a range of the data from the range bar below. And most importantly reset it all to dive into more data.

Updates outside of the control panel:

A lot of reforming to the back-end has happened in preparation of the control panel.

Security is always a concern when dealing with the online world. We have adjusted some security features. The control panel (unlike commercial panels today) will not actually be able to modify anything within the network per-say. What I mean by that is every command done through the control panel will actually hit a sub-system which allows or disallows access at that level. Which then hits the root system which can only run specific commands pre-written into the system. So unlike most control panels today where the code is on the forefront of the system, CLDMV’s back-end is segregated to many systems to prevent hacking. While data can be retrieved from the control panel anything which has to change the system will always be done through our set of sub-systems to insure stability and security.

Also it’s important to note there was a brief downtime of FTP log in and email receiving on 2014/05/31 at approx 5pm PST to 5:30pm PST. This was due to a mass restructuring of the back-end in order to support the new changes to come.

Log Parser V2

May 27, 2014

Nathaniel Hyson

Development Features Information Information Servers Upgrades Why CLDMV No Comments

Prior to version 2 of he log parser the logs were simply dumped into the database for the past 60 days every hour. It was planned that Users could view these logs instantly through a control panel. But unless you really knew what you were looking at this data was sort of useless.

So we came up with V2 of the Log Parser. This parser parses errors and website access into tables which gives a lot more information about the logs. For example I can pull up a number of how many 404 errors happened on any site hosted with CLDMV on any day in the past 60 days. As well as any redirect. If I want to view all the 404 or redirects I can do so. This functionality will be included in the control panel asap.

Why did we do this?

Simple answer really. Analytics services can provide you with how many visits your site gets and a ton of data about this. However they only work on pages where their code is present. Since we can track on the server level we can provide a lot more data. This data helps website owners make critical decisions and code fixes where they see trends in their software.

“Why am I not being indexed by [insert name of search engine]?”

We hear this question quite often. Simply put we do not control how search engines index websites. But we can show you (now) that your site is being crawled by their bots. V2 of the log parser is also logs if the visit was a bot or not.

To show an example:

domain	year	month	day	bot_visits	bot_unique_visits	visits	unique_visits	fourohfours	redirects
blog.cldmv.net	2014	5	27	20	3	30	5	0	0

The above results show this site’s basic stats for 2014-5-27. Visits and Unique visits are overall. So all we have to due to find human visits is subtract the bot visits from those numbers.

year	month	day	domain	timestamp	ip	real_ip	ssl	bot	ajax	resource	url	response_code	visit	referer	bytes	gzip_ratio	log_entry	user_agent	request_type	request_version	remote_user
2014	5	28	example.com	1401248811	66.249.73.71		1	1	0	php	/exampleurl.php	302	1		31		66.249.73.71 - - [28/May/2014:03:46:51 +0000] "GET /exampleurl.php HTTP/1.1" 302 31 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-" "-" "https" "-"	Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)	GET	HTTP/1.1

Above is an example of all the data available through our Log Parse V2.

Basic Control Panel and Three of the Five Themes Completed

May 25, 2014

Nathaniel Hyson

Billing Clients Development Features Information Servers Upgrades No Comments

Earlier this week we rolled out the basic control panel. At this point there isn’t much. Pretty much just displays current bans on the server and invoice data.

Three of the five themes (Default, Orange, Light) have been completed. Blue and Dark are still to come. As well as additional features inside the control panel and a better look inside there.

Some things to come are also domain name management and SSL management. These are two services which we plan to provide on top of the hosting and email services we currently provide.

Updates to Web Server

May 17, 2014

Nathaniel Hyson

Bugs Features Servers No Comments

There was a bug affecting a few clients where they could not delete files in the root of their html folders. This has been fixed. I also have a process running every day now to ensure this doesn’t happen again. The following files will server up a default file if they are not found in the html root folder:

robots.txt
favicon.ico
favicon.png

The default robots.txt only had one rule which is to tell all bots to limit their crawl rate to 1 page per second. Any client can place their own custom robots.txt or favicon.ico in their html root and that file will be served up instead. Some additional cleanup and security was done to the FTP service. Namely when you log into the FTP, clients will only see folders which you can actually do stuff in now. Some of which are just backup or log folders which clients can only (most of them) download the files.

Automatic Optimization of images

May 15, 2014

Nathaniel Hyson

Features Information Servers Why CLDMV No Comments

I was cleaning out my email folder yesterday and I came across this old email. While I have removed anything that would specify who this email is with the results are still proving of CLDMV’s systems. I’m also pretty sure this email was prior to the double compression PNG system we currently use now.


More good news.. Here is the report from google on image compression:
Losslessly compressing image/cache/data/ServiceRepairEvinrude-180x350.png could save 22.9KiB (57% reduction). 
Losslessly compressing image/cache/data/ServiceRepairMercury-180x350.png could save 21.2KiB (61% reduction). 
Losslessly compressing image/cache/data/free_shipping_smlbnnr-550x62.png could save 12.2KiB (61% reduction)

Here are the results from optimizing them in my system:
Processed file: image/cache/data/ServiceRepairEvinrude-180x350.png 10 times (60.49%).
Processed file: image/cache/data/ServiceRepairMercury-180x350.png 2 times (62.61%).
Processed file: image/cache/data/free_shipping_smlbnnr-550x62.png 2 times (62.39%).

More good news.. Here is the report from google on image compression:

Losslessly compressing image/cache/data/ServiceRepairEvinrude-180x350.png could save 22.9KiB (57% reduction).

Losslessly compressing image/cache/data/ServiceRepairMercury-180x350.png could save 21.2KiB (61% reduction).

Losslessly compressing image/cache/data/free_shipping_smlbnnr-550x62.png could save 12.2KiB (61% reduction)

Here are the results from optimizing them in my system:

Processed file: image/cache/data/ServiceRepairEvinrude-180x350.png 10 times (60.49%).

Processed file: image/cache/data/ServiceRepairMercury-180x350.png 2 times (62.61%).

Processed file: image/cache/data/free_shipping_smlbnnr-550x62.png 2 times (62.39%).

As you can see from above not only did our system compress the images as much as Google said we could. But we were able to compress 1-4% more. Here is a recent daily report of our image compression system.


[/www/blog.example.net/html] 230 files optimized at 48.35% overall rate.
[/www/example.net/html] 12 files optimized at 40.46% overall rate.

[/www/blog.example.net/html] 230 files optimized at 48.35% overall rate.

[/www/example.net/html] 12 files optimized at 40.46% overall rate.

Keeping in mind all the compression is done lossless. Meaning there is no loss in visual quality. The only downside to the compression is if the images store data such as where they were taken, that data is removed.

Minify HTML as a service

May 5, 2014

Nathaniel Hyson

Features Servers No Comments

While mileage may vary with minifying HTML it is not done automatically like minifying css and js. However there is a background service running for anyone wishing to use it on the server free of charge. Below is the php code to access the service.


function minifyhtml($html){
	$post = array(
		'html'	=> $html
	);
	$url = '127.0.0.1/minhtml.php';

	$ch = curl_init( $url );
	curl_setopt( $ch, CURLOPT_POST, 1);
	curl_setopt( $ch, CURLOPT_POSTFIELDS, http_build_query($_POST));
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
	curl_setopt($ch, CURLOPT_VERBOSE, 1);
	curl_setopt($ch, CURLOPT_HEADER, 1);
	$response = curl_exec( $ch );
	
	$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
	$header = substr($response, 0, $header_size);
	$body = substr($response, $header_size);
	$headers = array();
	$tmp = explode("n",trim($header));
	foreach($tmp as $line){
		if (trim($line) != ''){
			if (strstr($line,':')){
				$line = explode(':',$line);
				$key = trim($line[0]);
				unset($line[0]);
				$line = trim(implode(':',$line));
				$headers[$key] = $line;
			}
		}
	}
	return array($body,$headers);
}

function minifyhtml($html){

$post = array(

'html' => $html

);

$url = '127.0.0.1/minhtml.php';

$ch = curl_init( $url );

curl_setopt( $ch, CURLOPT_POST, 1);

curl_setopt( $ch, CURLOPT_POSTFIELDS, http_build_query($_POST));

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($ch, CURLOPT_VERBOSE, 1);

curl_setopt($ch, CURLOPT_HEADER, 1);

$response = curl_exec( $ch );

$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);

$header = substr($response, 0, $header_size);

$body = substr($response, $header_size);

$headers = array();

$tmp = explode("n",trim($header));

foreach($tmp as $line){

if (trim($line) != ''){

if (strstr($line,':')){

$line = explode(':',$line);

$key = trim($line[0]);

unset($line[0]);

$line = trim(implode(':',$line));

$headers[$key] = $line;

}

return array($body,$headers);

}

Combined with the above code you can use the below code to retrieve the minified html body as well as the original bytes, minified bytes, and saved percent by minifying the html.


list($body,$headers) = minifyhtml($src);
$srcBytes = $headers['X-SRCBytes'];
$minOutputBytes = $headers['X-OutputBytes'];
$savedPercent = $headers['X-SavedPercent'];

list($body,$headers) = minifyhtml($src);

$srcBytes = $headers['X-SRCBytes'];

$minOutputBytes = $headers['X-OutputBytes'];

$savedPercent = $headers['X-SavedPercent'];

Crons are here

April 16, 2014

Nathaniel Hyson

Features Servers No Comments

While there is still no control panel yet. The system now supports schedules crons. These crons can be set at a min interval of 5 minutes and are easily modified. So once the control panel is in place it will be easy to implement it into the panel.

Currently I have already setup the crons manually that were needed for some clients. If you need any crons setup for now just send me an email.

Note: These crons have a low priority in order to keep the server running smoothly. (nice commanded)

PHP CLI added, crons to come soon

April 9, 2014

Nathaniel Hyson

Features Servers No Comments

PHP has been added to the shells. There was a bug with the mysql side of this that didn’t let PHP load the mysql extensions. Now that that is fixed PHP can properly be used through the command line for each user.

With this being fixed cron jobs will be the second thing on my list to implement into the control panel.