Category Archives: Development

New Testing for speed comparisons

While we still continue to compare new client’s sites with Pagespeed. We have also put together a tool which will show us aggregated time/count/sizes of loaded websites. Below is a read out of latest as well as Pagespeed results:

 

Site  Response Time  Load Time  Download (First Load)  Page Speed Score  Cost
Original340ms3.34s4.24MB  64 /   50?
Compare300ms1.38s3.5MB  88 /   70150$~

Updates

It’s been some time since a post was made here on our blog. We have been extremely busy with a couple large projects. So busy we are actually expanding our office.

While we can’t release a whole lot of information at this time. We are also excited to announce that we will be Writing a modularized platform software package. From this package users will be able to add in blogs, forums, shopping carts, etc. All into the same system and all able to run multiple domains. While this will be no easy task and it will take some time to complete. We believe there is a great need for software of this calibre.

Right now there are great softwares out there for each individual task. But then you have to mesh them together, spend thousands of dollars getting them to look the same. Even then they still don’t work together they work alongside each other. A user on your forum can’t log into your shopping cart, same is true with the blog.

More about this project will be announced in the coming months as licensing and trademarking of the name are complete.

DKIM Signing and Mail filtering

Spam is always a concern when you have email. We have successfully integrated DKIM signing to our email system. As it can break mail delivery this is not done automatically. DNS records need to be in place for it to work properly. Thus if you’re hosted at CLDMV please contact us in order to help you with the process of setting up DKIM.

Next upgrade we have put in place is our own custom mail filter. Right now there isn’t much to it but injecting a header into emails to show it working. But soon there will be SMTP Relay limits (used to prevent compromised emails from flooding emails outbound), Custom spam filtering though DB rules, and probably some other things which I’m not thinking of currently.

The two main issues we had with putting our own Filter in place was the ability to maintain our Anti-Virus and Spam systems that were already in place while also being able to provide a second content filter. After many hours and tons of caffeine we were able to produce a system which kept our current system intact while adding the secondary system on top of it. As always Security in this manner was a great concern. Especially considering we would be handling external content. Several limitations are at play in current mail systems which caused us to create innovate techniques in order to send emails to our own system securely. We even attempted to hack our own mail server through emails and were unsuccessful.

Current Status of Control Panel, security updates, and server updates

First lets start with a screen shot of the dashboard in progress for the control panel.

current-cp

Unfortunately Data prior to the 28th of May 2014 is a bit skewed. But as you can see we have plenty of navigation so consumers never get lost. We also show are currently showing all site traffic for all sites hosted under a specific user. Though with some crashes here and there we might put these charts on their own separate page per domain but still give the user the ability to view an all domain report as it’s setup right now (at their own risk).

Currently the theming for the charts is still to be completed. But we have the ability to show and hide specific points of data if we want, select a range of the data from the range bar below. And most importantly reset it all to dive into more data.

Updates outside of the control panel:

A lot of reforming to the back-end has happened in preparation of the control panel.

Security is always a concern when dealing with the online world. We have adjusted some security features. The control panel (unlike commercial panels today) will not actually be able to modify anything within the network per-say. What I mean by that is every command done through the control panel will actually hit a sub-system which allows or disallows access at that level. Which then hits the root system which can only run specific commands pre-written into the system. So unlike most control panels today where the code is on the forefront of the system, CLDMV’s back-end is segregated to many systems to prevent hacking. While data can be retrieved from the control panel anything which has to change the system will always be done through our set of sub-systems to insure stability and security.

Also it’s important to note there was a brief downtime of FTP log in and email receiving on 2014/05/31 at approx 5pm PST to 5:30pm PST. This was due to a mass restructuring of the back-end in order to support the new changes to come.

Log Parser V2

Prior to version 2 of he log parser the logs were simply dumped into the database for the past 60 days every hour. It was planned that Users could view these logs instantly through a control panel. But unless you really knew what you were looking at this data was sort of useless.

So we came up with V2 of the Log Parser. This parser parses errors and website access into tables which gives a lot more information about the logs. For example I can pull up a number of how many 404 errors happened on any site hosted with CLDMV on any day in the past 60 days. As well as any redirect. If I want to view all the 404 or redirects I can do so. This functionality will be included in the control panel asap.

Why did we do this?

Simple answer really. Analytics services can provide you with how many visits your site gets and a ton of data about this. However they only work on pages where their code is present. Since we can track on the server level we can provide a lot more data. This data helps website owners make critical decisions and code fixes where they see trends in their software.

“Why am I not being indexed by [insert name of search engine]?”

We hear this question quite often. Simply put we do not control how search engines index websites. But we can show you (now) that your site is being crawled by their bots. V2 of the log parser is also logs if the visit was a bot or not.

To show an example:

domainyearmonthdaybot_visitsbot_unique_visitsvisitsunique_visitsfourohfoursredirects
blog.cldmv.net201452720330500

The above results show this site’s basic stats for 2014-5-27. Visits and Unique visits are overall. So all we have to due to find human visits is subtract the bot visits from those numbers.

yearmonthdaydomaintimestampipreal_ipsslbotajaxresourceurlresponse_codevisitrefererbytesgzip_ratiolog_entryuser_agentrequest_typerequest_versionremote_user
2014528example.com140124881166.249.73.71110php/exampleurl.php30213166.249.73.71 - - [28/May/2014:03:46:51 +0000] "GET /exampleurl.php HTTP/1.1" 302 31 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-" "-" "https" "-"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)GETHTTP/1.1

Above is an example of all the data available through our Log Parse V2.

Basic Control Panel and Three of the Five Themes Completed

Earlier this week we rolled out the basic control panel. At this point there isn’t much. Pretty much just displays current bans on the server and invoice data.

Three of the five themes (Default, Orange, Light) have been completed. Blue and Dark are still to come. As well as additional features inside the control panel and a better look inside there.

Some things to come are also domain name management and SSL management. These are two services which we plan to provide on top of the hosting and email services we currently provide.

Developer Frustration on CLDMV

Recently I’ve been putting a lot of time into development of the control panel. Development of the control panel had been hindered though by one system that makes the sites on CLDMV’s network run faster.

OPCache was the problem. This module of PHP cache’s the results of the compiled PHP code and then serves that up if there is a cache rather than re-compiling the PHP code again. Seems rather simple and it was. For future reference for any developer working on a site in CLDMV’s network. All you have to do is add the below snippet of code to the first file that is being called to disable the cache during development stages.