Difference between revisions of "Main Page"

From wiki.mikejung.biz
Jump to navigation Jump to search
(Install Zip from source)
(Collectible Trading Cards)
 
(567 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==Links!==
+
<seo title="wiki.mikejung.biz" titlemode="append" keywords="linux wiki,lamp wiki,Mike Jung"  description="A wiki created and maintained by Mike Jung. The wiki covers most of the LAMP stack,cPanel, as well as Windows tuning with a focus on performance and optimization of all the things"></seo>
  
http://wiki.churchoftheinterwebz.com/index.php?title=Links
+
==Wiki Landing Page==
 +
Hello there! My name is Mike Jung and this is my wiki. I hope some of the information here will help someone out or teach them something new.
  
 +
I realize you are likely here for technical content, which is exactly why you should check out these scanned vintage '''[[Real photo postcards]] from the 1900s''', some of them are kinda cool actually!
  
  
 +
== Wiki Navigation and Page List ==
  
== "Can you take a look at my server and tell me what's going on?"==
+
===Collectible Trading Cards===
  
===Starting Points===
+
This section covers various collectable trading cards from the 1990s and 2000s as well as collectable trading card games such as Pokemon, Yu-Gi-Oh and Magic the Gathering. Many of these artists have done work for major comic book companies, movies, tv shows and other forms of media that have enhanced our lives over the years. 
  
'''Who is on the server?'''
+
*[[Mike Ploog]] - Mike Ploog is a famous comic artist that has worked on countless movies, comics, tv shows and other works.
<pre>
+
 
w
+
*[[Tim and Greg Hildebrandt]] - These two brothers have created Star Wars movie posters, drawn countless X-Men, worked on LOTR and countless other epic things, check out their work!
</pre>
+
 
 +
*[[Lady Death]] - Very badass stuff!
 +
 
 +
*[[1992 Boris Series 2 Trading Cards]] - Adult Fantasy Art. Mind blowing to say the least!
 +
 
 +
*[[Boris 3 Prism Trading Cards]] - The 3rd series from Boris Vallejo, featuring epic art and prism colors!
 +
 
 +
*[[Yu-Gi-Oh! Trading Cards]] - Check out these rare and cool looking cards!
 +
 
 +
*[[1993 SkyBox DC Cosmic Teams]] - One of my favorite trading card series from DC / SkyBox from the early 1990s, check out the cool holograms!
 +
 
 +
*[[Deathwatch2000 Collectible Trading Cards]] - Excellent sci fi series from 1993, brutal!
 +
 
 +
*[[Mighty Morphin Power Rangers]] - If you're a millennial, surely you must know about the power rangers!
 +
 
 +
===Hearthstone===
 +
 
 +
*[[Legendary Ramp Druid]] - If you like Hearthstone, Legendary cards, and playing a Druid, visit this page!
 +
 
 +
===Windows 8.1 Utilities and Windows Tech Preview 10===
 +
 
 +
*[[MPC-HC Audio Configuration]] - While this page focuses on configuring MPC-HC for the best audio quality, it can still be used to configure Windows 8.1 audio for high end sound cards and speakers such as the Logitech Z 906 speakers and the Sound Blaster Z sound card.
 +
 
 +
*[[Windows 10 Tech Preview]] - Currently testing out Windows 10 Tech Preview, Build 9926. This wiki page will eventually have tons of information and updates on Windows 10.
 +
 
 +
*[[CrystalDiskMark]] - A beginner guide on how to use CrystalDiskMark to benchmark your HDD or SSD.
 +
 
 +
*[[S3 Browser]] - S3 Browser is similar to an FTP client, but it speaks to a REST endpoint of an S3 compatible Object Storage service. AWS S3 is supported, but other compatible Object Storage services are also supported.
 +
 
 +
*[[MSI Afterburner]] - MSI Afterburner configuration guide / walk through with screenshots that explain how to do stuff with afterburner, like select GPU usage in OSD.
 +
 
 +
*[[Virtualbox]] - Various tips and tricks relating to VirtualBox, which allows you to run virtual machines on Windows hosts.
 +
 
 +
=== GPU and Video Decoding Stuff (madVR, CUDA, MPC-HC) ===
 +
 
 +
*[[How_to_use_Geekbench_on_Windows_10|Geekbench4]] - How to use GeekBench4 to measure CPU and GPU compute performance.
 +
 
 +
*[[Nvidia Noise Reduction]] - Nvidia Noise Reduction reduces noise during video playback, how much does this improve quality? Not a ton, in some cases, anyway, check out the page for more info.
 +
 
 +
*[[Nvidia Edge Enhancement]] - Ever wonder if the Nvidia Video settings make much of a difference in terms of video quality? Turns out it can make a huge difference! This page focuses on Nvidia Edge Enhancements
 +
 
 +
*[[MPC-HC Video Decoder Comparison]] - Screenshot comparisons between MPC-HC's default Video Decoder (DXVA) and CUVID. If you have a Nvidia GPU and want to make sure you have the best video quality, check out this page!
 +
 
 +
*[[DyingLight]] - Some PC screenshots of DyingLight, using Nvidia DSR with a GTX 970. DSR does improve quality, but is very costly in terms of performance.
 +
 
 +
*[[MadVR Image Doubling 720p]] - 720p madVR Image Doubling screenshot comparison page for 720p video playback
 +
 
 +
*[[Windows 8.1 MPC-HC and MadVR Setup Guide]] - A Windows 8.1 based guide on how to properly configure MPC-HC (media player classic home cinema) to work with MadVR as the video renderer. This setup guide includes images for each of the steps and explanations of the main mpc-hc and madvr settings.
 +
 
 +
*[[CUDA]] - GPUs are on track to control the global population. CUDA is how the matrix started. It's pretty cool though, CUDA allows a GPU to accelerate some types of processing that previously only the CPU could compute.
 +
 
 +
*[[DXVA2]] - A quick article that explains what DXVA2 is and how it interacts with a GPU during the video playback process.
 +
 
 +
*[[MadVR]] - MadVR MPC-HC wiki containing optimization tips and benchmarks for Chroma Upscaling, Image Doubling, Image Upscaling, Image downscaling and many other configuration settings.
 +
 
 +
*[[PotPlayer]] - PotPlayer wiki about how to install, configure and optimize potplayer using MadVR, CUDA and GPU magic
 +
 
 +
*[[PotPlayer Advanced Configuration]] - Slightly more detailed than the main PotPlayer wiki.
 +
 
 +
*[[DirectShow]] - What is directshow? What does directshow do? Want to learn more about directshow? Then please visit this page!
 +
 
 +
*[[MadVR Chroma Upscaling]] - MadVR Chroma Upscaling performance results and general information on the best scaling algorithm to use to upscale Chroma with MadVR.
 +
 
 +
*[[Error Diffusion Dithering]] - madVR includes some advanced dithering options which utilize a DX 11 GPU.
 +
 
 +
*[[MadVR Chroma Upscaling 720p Image Quality]] - Screenshots of all the madVR Chroma Upscaling options using a 720p video
 +
 
 +
*[[MadVR Chroma Upscaling 1080p Image Quality]] - Screenshots of all the madVR Chroma Upscaling options using a 1080p video
 +
 
 +
*[[MadVR Image Doubling 720p]] - Screenshots of all the madVR Resolution Doubling options using a 720p video
 +
 
 +
*[[MadVR Image Upscaling]] - Screenshots of all the madVR Image Upscaling options using a 720p video
 +
 
 +
*[[MadVR Processing Settings]] - Guide that covers some of the madVR processing settings
 +
 
 +
*[[Nnedi3]] - madVR recently added the ability to use NNEDI3 to improve video quality, this wiki covers how to configure this.
 +
 
 +
===cPanel Stuff===
 +
 
 +
*[[EasyApache 4]] -- cPanel EasyApache 4 guide
 +
 
 +
*[[CloudLinux]] -- Overview on what CloudLinux is and the types of resources that it limits for cpanel users.
 +
 
 +
*[[CPanel]] - Main cPanel wiki
 +
 
 +
*[[PHP Handlers]] - How to change cPanel's main PHP handler via CLI and via WHM.
 +
 
 +
*[[Whmcs plugin|LiquidWeb WHMCS Plugin]] - A FREE WHMCS plugin created by [http://www.modulesgarden.com/ ModulesGarden] that allows anyone to resell SSD VPS servers, or whatever kind of servers you want. Basically this plugin is a Cloud Business in a box and it's free.
 +
 
 +
*[[CPanel Statistics Software]] - Learn how to configure cPanel to process logs and bandwidth during low traffic hours of the day. This reduces the amount of Apache restarts.
 +
 
 +
*[[Whmcs 6]] - Need help installing and configuring WHMCS 6? Visit this page!
 +
 
 +
*[[Ioncube loader]] - Are you getting ioncube loader errors or encountering version issues? Visit my ioncube loader page to learn more!
 +
 
 +
=== Webserver Stuff ===
 +
 
 +
*[[Litespeed]] - Information about the litespeed webserver installation process and how to correctly configure litespeed on a cpanel server.
 +
 
 +
*[[Apache]] - Do you like websites? You can thank Apache! It's the most common webserver around. Nginx is gaining some steam, but Apache is still pretty awesome!
 +
 
 +
*[[GooglePageSpeed]] - How to install and configure the mod_pagespeed module with Apache, in addition, and really importantly, How to install Google's mod_pagespeed on a cPanel server via Easy Apache. lol
 +
 
 +
*[[WordPress Optimization]] - Guide on how to optimize a server for wordpress. Includes Apache Event configuration, PHP-FPM configuration, MySQL configuration and lots more!
 +
 
 +
*[[Application Optimization]] - General website optimization guide
 +
 
 +
=== MySQL, PHP and Caching ===
 +
 
 +
*[[PHP 7]] - This wiki page is still being worked on, but it will focus specifically on PHP 7 and how awesome it is.
 +
 
 +
*[[PHP MySQL Extensions]] - Also still in the works, will focus specifically on the MySQL extensions for PHP.
 +
 
 +
*[[Memcached]] - Caching makes everything faster! I like fast things, so I use memcached a lot and you should use memcached too! I'll show you how to use memcached to improve website load time and reduce latency when connecting to a database! All of this can be done if you know how to tame the mythical beast know as memcached.
 +
 
 +
*[[PHP_OPcache]] - Do you like fast things? Want to make PHP faster? Use opcode caching. For your health!
 +
 
 +
*[[fcgid]] - FastCGI will make ur blog faster! Maybe, if you know how to configure Apache to use FastCGI to proxy PHP requests to a dedicated PHP process! If you want to learn more about using the FCGI handler on cpanel, please check out this page!
 +
 
 +
*[[Php-fpm]] - Speaking of awesome...PHP-FPM is here. Are you still using mod_php and wondering why apche is slow? It's because you are doing php wrong! Check out this page for information on how to install, tune, and optimize php-fpm with apache
 +
 
 +
*[[Varnish]] - Varnish can be an excellent way to improve your website's response times by caching the most used static content, like images, varnish is able to significantly reduce the amount of work that apache has to do.
 +
 
 +
*[[MySQL]] - Learn all about the various my.cnf settings for MySQL 5.5 and MySQL 5.6. This page is less focused on optimizations, and more focused on covering the best practices for configuring a server to handle MySQL.
 +
 
 +
*[[MediaWiki Performance Tweaks]] - I use MediaWiki, if you can't tell already. I've created a page that lists all the caching settings / configurations to make this site somewhat fast.
 +
 
 +
*[[HHVM]] - Facebook's attempt to take PHP to the next level. HHVM uses a JIT compiler to significantly reduce the amount of resources it takes to interpret and run a PHP script.
 +
 
 +
=== Monitoring and Analysis ===
 +
 
 +
*[[Cloud Performance Tuning]] - High response times got you bummed out? Slow website? Fear not!
 +
 
 +
*[[Newrelic]] - Newrelic is pretty awesome. They offer a free tier which lets you monitor server resources for 24 hours. You can also utilize APM which is an application monitoring services which shows the response time if your application and database. If you are looking for common Newrelic agent commands or need help troubleshooting Newrelic's agents, check out this page!
 +
 
 +
*[[Sysdig]] - Looking for a utility that will provide insight into application and Linux performance? Sysdig is your tool! I really like it and find it pretty useful so I made a wiki!
 +
 
 +
*[[Sysstat]] - Sysstat contains sar which is used to record server resource usage over the course of each day. Sar is really helpful if you care about server performance so knowing how to view data like swap in and swap out activity is critical.
 +
 
 +
=== Browser and Front End ===
 +
 
 +
*[[Chrome]] - A list of tweaks (flags) that you can enable in the Chrome and Chromium web browsers which can help to speed up performance. Useful if you notice slow, laggy websites and want to speed up your browser.
 +
 
 +
*[[HTTP 2.0]] - Still in creation mode, this wiki will eventually contain all kinds of information on the new HTTP 2.0 protocol.
 +
 
 +
*[[Browser Caching]] - .htaccess examples for how to enable browser caching of images and static files.
 +
 
 +
=== Benchmarking and Performance Tuning Stuff ===
 +
 
 +
*[[Cloud Performance Tuning]] -- Learn how to diagnose and fix performance issues.....in the cloud!
 +
 
 +
*[[Benchmarking]] -- A linux benchmarking reference wiki with many example commands and explanations for sysbench, fio, iozone and ioping tests.
 +
 
 +
*[[Sysbench]] -- Similar to the benchmarking wiki but with 100% focus on sysbench and how to benchmark vps and cloud servers.
 +
 
 +
*[[Google PerfKitBenchmarker]] -- How to install, configure and run benchmarks with Google PerfKitBenchmarker.
 +
 
 +
*[[Phoronix Test Suite]] -- How to install Phoronix Test Suite on CentOS and Ubuntu. Examples on how to use the test suite and an overview on what each test does.
 +
 
 +
*[[OS Tuning]] - You can't tune an application until you tune the operating system. Check out my OS system tuning wiki for tips and tricks on speeding up your slow CentOS server.
 +
 
 +
*[[Performance Troubleshooting Methodologies]] - How to analyze resource usage on a Linux box. 
 +
 
 +
*[[Dmcache]] - Caching, SSDs, what could be better? What about using SSDs to cache your slow as balls hdds? Learn how to by checking out this dmcache wiki!
 +
 
 +
*[[NUMA]] - http://wiki.mikejung.biz/NUMA
 +
 
 +
*[[Siege]] - Learn how to compile siege so you can test website performance on http and https websites
 +
 
 +
*[[VMware Workstation Player Tuning]] - Want to learn more about VMware player's configurations settings?
 +
 
 +
=== Storage and File System Stuff ===
 +
 
 +
*[[LVM Commands]] - LVM command reference guide. Explains what logical volumes and logical groups are all about and how to create an LVM volume
 +
 
 +
*[[Ceph]] - Ceph is a distributed storage system that powers the open cloud and internet of things. Just kidding, it doesn't do all that but it is still pretty awesome technology!
 +
 
 +
*[[LSI]] - LSI makes RAID cards. Been around for a long time, recently bought out by seagate. LSI cards are nice, but sometimes slow if you do not configure RAID for performance. If you want to add some performance to your RAID, make sure you configured the card correctly!
 +
 
 +
*[[DRBD]] - Data replicating block device, aka DRBD has been a heavyweight in the cloud storage wars for a while now. You got Ceph in one corner, OCFS2 in another, RAID (for backups) and DRBD. DRBD can be tricky to configure and even if you get it to work it might still be somewhat slow. I have created a wiki that covers some basic performance tuning for DRBD.
 +
 
 +
*[[Big Data]] - Main page that links to topics like Cassandra and Hadoop.
 +
 
 +
===Linux Kernel Stuff===
 +
 
 +
*[[Sysctl tweaks]] - hackin yo kernel with sysctl
 +
 
 +
*[[Linux documentaries]] - Page with links to various Linux and Unix documentaries
 +
 
 +
*[[Software RAID]] - How to use mdadm and configure software RAID on Linux
 +
 
 +
*[[Ubuntu]] - General wiki for how to do things on Ubuntu.
 +
 
 +
*[[Ubuntu Performance Tuning]] - Ubuntu performance tuning page, how to optimize for an SSD
 +
 
 +
=== Other Stuff ===
 +
 
 +
*[[How to install Glibc-2.15 on CentOS6]] - CentOS 6 has old versions of everything, especially glibc. This guide shows you how to install an additional new version! 
 +
 
 +
*[[Firewalld]] - Are you used to iptables and hate firewalld? It's ok, I feel your pain! Firewalld is actually pretty easy to use so if you're running CentOS 7 or Redhat 7 please visit this page.
 +
 
 +
*[[Hearthstone Combo Guide]] - I'm a Hearthstone nub, so I made a wiki on how to build a deck that will at least give you some options when it comes to minion combinations.
 +
 
 +
*[[Real photo postcards]] - Totally random old picture post card wiki. Completely unrelated to anything that has to do with the internet.
 +
 
 +
*[[CentOS 7]] - Main page for CentOS 7 which includes all kinds of performance tweaks for the 3.10 kernel or newer.
 +
 
 +
*[[Load Balancing]] - Learn more about the Stingray / Riverbed Traffic Manager! It's pretty cool and has a ton of options, if you are looking for some load balancing information, check out the wiki!
 +
 
 +
*[[How to save bash directory location]]  - Bashmarks Is a simple tool that allows you to save directory locations and then later return to them using extremely simple commands that even tab complete!
 +
 
 +
*[[Cassandra]] - Cassandra is a NOSQL like DB that Apache made. This wiki contains general information about what Cassandra is, how it works and details on the topology.
 +
 
 +
*[[Hadoop]] - Also NoSQL like, hadoop is great for running batch jobs against a large amount of data.
 +
 
 +
*[[Gcc CentOS]] - Why is GCC always old on CentOS? Why does CentOS always ship old software? I do not know, but I can show you how to update GCC on CentOS if you visit the GCC CENTOS wiki!
 +
 
 +
*[[MySQL Optimization]] - How to optimize MySQL
 +
 
 +
*[[ISCSI Commands]] - Some commonly used Linux ISCSI commands.
 +
 
 +
*[[Networking]] - Linux Networking wiki
 +
 
 +
*[[Hardware]] - Wiki that covers SSDs, PCIe Flash SSDs, RAID cards and other types of server hardware. Includes tips and tweaks to improve performance and stability.
 +
 
 +
*[[Security]] - Basic ways to secure your server and website. How to track down malicious files and how to clean up a hacked website.
 +
 
 +
*[[R1soft]] - Idera / R1soft backup manager overview
 +
 
 +
*[[Processor]] - Basic information about CPUs and how they work
  
'''Show top processes.'''
+
*[[ApacheTheory]] - Information about how Apache works, how to optimize Apache and an explanation on the role of Apache.  
<pre>
 
top -c
 
</pre>
 
  
'''Load averages.'''
+
*[[Logs]] - A list of where the common cPanel log locations are and what type of information they hold.
<pre>
 
sar -q
 
</pre>
 
  
'''Ram usage.'''
+
*[[Sysstat]] - sysstat includes sar, which is awesome
<pre>
 
sar -r
 
</pre>
 
  
'''I/O wait'''
+
== for loop example script ==
<pre>
 
sar -s
 
</pre>
 
  
After you get a good idea of what is causing load / unresponsiveness, you can start narrowing things down.
+
*[[How to create a for loop script]] If you want to become a linux wizard and master the art of for loops, check out this page!
  
===Apache===
+
== Sed ==
  
'''Apache Status'''
+
'''Add a word to the begining of a line'''
 
<pre>
 
<pre>
/usr/bin/lynx -dump -width 500  http://127.0.0.1/whm-server-status | less
+
sed 's/^/$Wordtoadd/' original.txt > sorted_original.txt
 
</pre>
 
</pre>
  
'''Apache connection'''
+
== Linux commands to check for DDoS and excessive connections ==
<pre>
+
 
/usr/bin/lynx -dump -width 500 http://127.0.0.1/whm-server-status | awk '{print $11" "$12}'| awk NF |grep [0-9].[0-9].[0-9].[0-9]|sort|uniq -c|sort -n|tail -50
+
This section is being merged into the main Apache wiki which can be found '''[http://wiki.mikejung.biz/Apache#How_to_check_for_excessive_connections_to_Apache here]'''
</pre>
 
  
'''Check settings in httpd.conf. Added +160 usually located around that line number.'''
+
== Web Permissions | Files ==
 +
'''Default Web Permissions NOTE: MAKE SURE YOU ARE IN A public_html directory!!!!!!'''
 
<pre>
 
<pre>
vim /usr/local/apache/conf/httpd.conf +160
+
find . -type f -exec chmod 644 {} \;
 +
find . -type d -exec chmod 755 {} \;
 
</pre>
 
</pre>
  
'''Think Apache is causing server to go OOM? Check PHP memory limit. If it's above 32M ask client if they need it this high.'''
 
<pre>
 
grep memory_limit /usr/local/lib/php.ini
 
</pre>
 
  
 
'''Find all users php.ini files.'''
 
'''Find all users php.ini files.'''
Line 64: Line 281:
 
</pre>
 
</pre>
  
===MySQL===
+
== Modules / Directives ==
  
'''Useful for seeing what queries are doing what.'''
+
'''speling'''
 +
 
 +
mod_speling.c
 +
 
 +
Once added via Easy Apache, you can simply add these directives to a .htaccess file
 
<pre>
 
<pre>
watch -n 1 'mysqladmin proc stat'
+
CheckCaseOnly On
 +
CheckSpelling On
 
</pre>
 
</pre>
  
'''Check /tmp for sess_* files. Can mean tables are corrupt.'''
+
==How To Optimize WordPress==
 +
 
 +
For a detailed guide, please visit my [[http://wiki.mikejung.biz/index.php?title=WordPress_Optimization WordPress Optimization Guide]]
 +
 
 +
== PHP ==
 +
 
 +
Install ssh2 Pecl extension
 +
 
 
<pre>
 
<pre>
ll /tmp/
+
yum install libssh2 libssh2-devel
</pre>
+
pecl install ssh2
 +
# You may need to update the channel, if so;
 +
pecl channel-update pecl.php.net
  
'''Check the logs.'''
+
Then just;
<pre>
+
vim /etc/php.ini
/var/lib/mysql/examplehostname.err
+
extension=ssh2.so
 
</pre>
 
</pre>
  
 +
Upload issues
  
'''Then check MySQL settings'''
+
May need to check two files, the first is the global php.ini file, the next is the modsec file (if applicable)
 
<pre>
 
<pre>
vim /etc/my.cnf
+
vim /usr/local/lib/php.ini
</pre>
+
 
 +
upload_tmp_dir = /tmp
 +
session.save_path = /tmp
  
== Network Stuff ==
+
----------------------------------------------
  
New server missing some IPs it was supposed to come with?
+
vim /usr/local/apache/conf/modsec2/custom.conf
  
<pre>
+
SecUploadDir /tmp
service ipaliases restart
+
SecTmpDir /tmp
 
</pre>
 
</pre>
  
<pre>
+
Parse Error
/scripts/rebuildippool
+
 
</pre>
+
Parse error: syntax error, unexpected T_STRING
 +
 
 +
Check the file and remove <?xml version="1.0" encoding="utf-8"?>
  
  
 +
Force PHP5
  
 +
Add to .htaccess:
  
== MySQL ==
+
AddType application/x-httpd-php5 .html .htm
  
=== One Liners ===
+
== Email ==
  
See MySQL status. Updates every 1 s.
+
How to enable DKIM for a cpanel account
  
<pre>
+
*DomainKeys Identified Mail (DKIM) defines a mechanism by which email messages can be cryptographically signed, permitting a signing domain to claim responsibility for the introduction of a message into the mail stream. Message recipients can verify the signature by querying the signer's domain directly to retrieve the appropriate public key, and thereby confirm that the message was attested to by a party in possession of the private key for the signing domain.
watch -n 1 'mysqladmin proc stat'
 
</pre>
 
  
Optimize Tables.
+
*To verify that everything is setup correctly you can send an email from an email account on that domain to [email protected] No need to have a subject or body. This service will then reply with a message stating the verification of DKIM, DomainKeys, SPF, SpamAssassin, and Sender-ID. Great tool to test all kinds of email verification systems.
  
 +
To install on a cPanel server:
 
<pre>
 
<pre>
for i in $(mysql -e "show databases;" | sed 's/Database//') ; do for each in $(mysql -e "use $i; show tables;" \
+
/usr/local/cpanel/bin/dkim_keys_install <username>
| sed 's/Tables.*//' ;) ; do mysql -e "use $i ; optimize table $each" ; done ; done
 
</pre>
 
  
MySQL check that will: check all databases, analyze, optimize and repair. Pretty useful, usually safe.
+
or
  
<pre>
+
for i in `ls /var/cpanel/users`; do /usr/local/cpanel/bin/dkim_keys_install $i; done
mysqlcheck -Aaor
 
 
</pre>
 
</pre>
  
Dump a database.
+
*Add the Policy Record
 
<pre>
 
<pre>
mysqldump database > database.sql
+
_domainkey    IN    TXT    "t=y; o=~; n=Interim Sending Domain Policy; [email protected].com"
 
</pre>
 
</pre>
  
Import a database.
+
'''General webmail and email permission guidelines for cPanel servers'''
 +
 
 +
Below are some baseline permissions that should be used with Exim and Dovecot:
 +
 
 +
'''/home/user/etc/'''
 
<pre>
 
<pre>
mysql database < database.sql
+
domain.com file should have:
 +
permissions:
 +
750
 +
ownership:
 +
username:mail
 
</pre>
 
</pre>
  
Connect to a database.
+
'''/home/user/etc/domain.com/'''
 
<pre>
 
<pre>
mysql -u user -h ip -p databasename
+
passwd
 +
permissions:
 +
640
 +
ownership:
 +
user:mail
 +
 
 +
quota
 +
permissions:
 +
640
 +
ownership:
 +
user:mail
 +
 
 +
shadow
 +
permissions:
 +
640
 +
ownership:
 +
user:user
 
</pre>
 
</pre>
  
===Viewing and Deleting Tables and Databases===
+
'''/home/user/mail/'''
Look at databases and tables.
 
 
 
 
<pre>
 
<pre>
mysql
+
700 user:user cur/
> use databasename;
+
751 user:user domain.com/
> show tables;
+
700 user:user anything else
 
</pre>
 
</pre>
  
Drop (delete) a database. Can be useful if importing a database and it gives you an error.
+
If email accounts are not showing up in cPanel for a specific cpanel user be sure to check '''/home/$user/etc''' to make sure the passwd file and shadow file have proper permissions also make sure they are located in
 
 
 
<pre>
 
<pre>
mysql
+
/home/user/etc/domain.com/
> drop database databasename;
 
 
</pre>
 
</pre>
  
=== Optimization Scripts ===
+
If all the permissions are correct and the directories are owned by the user, try restarting cpanel mail services to see if this helps resolve the issue.
  
While the default configs here are a good starting point. These scripts will help in finding any issues with the users current MySQL config.
 
  
Note: I like to run these like : '/scripts/tuning-primer.sh > /root/tuning-primer.txt[n]' This saves the output, so you don't
+
If you run into a Roundcube error like "unable to connect to database", the best thing to do is to drop the database, then re-install roundcube, which automatically re-creates the db. '''Make sure you backup the database before you drop it, or else you risk lots of possible data loss'''
feel compelled to add it as a note to a ticket or admin comments.  You can also use it to compare the results after 48 hours.
+
<pre>
This is a great way to document these changes.
+
cd /home/temp
 +
mysqldump roundcube > roundcube.sql
 +
mysql -e "drop database roundcube;"
 +
/usr/local/cpanel/bin/update-roundcube --force
 +
</pre>
  
wget -O /scripts/tuning-primer.sh http://day32.com/MySQL/tuning-primer.sh
 
chmod +x /scripts/tuning-primer.sh
 
/scripts/tuning-primer.sh
 
  
If day32.com is down, try:
+
If you are running into spam issues you can run the command below to find top sending IPs in exim logs:
 +
<pre style="white-space: pre-wrap;
 +
white-space: -moz-pre-wrap;
 +
white-space: -pre-wrap;
 +
white-space: -o-pre-wrap;
 +
word-wrap: break-word;">
 +
grep "SMTP connection from" /var/log/exim_mainlog |grep "connection count" |awk '{print $7}' |cut -d ":" -f 1 |cut -d "[" -f 2 |cut -d "]" -f 1 |sort -n |uniq -c | sort -n
 +
</pre>
  
wget -O /scripts/tuning-primer.sh http://mysql-tuner.didfor.me
+
Find authenticated users who may be spamming:
+
<pre>
 +
find /var/spool/exim/input/ -name '*-H' | xargs grep 'auth_id'
 +
</pre>
  
wget -O /scripts/mysqltuner.pl http://mysqltuner.com/mysqltuner.pl
+
Spam comming from scripts:
chmod +x /scripts/mysqltuner.pl
+
<pre>
/scripts/mysqltuner.pl
+
grep cwd=\/home\/ /var/log/exim_mainlog| cut -d' ' -f4 | sort | uniq -c | sort -n
 +
</pre>
  
 +
Removing all queued messages at once in a safe way:
 +
<pre>
 +
exim -bp | awk '/^ *[0-9]+[mhd]/{print "exim -Mrm " $3}' | sh
 +
</pre>
 +
Or you can do the same from the mail queue manager in WHM.
  
===Upgrading MySQL===
 
{{Box Warning| '''If they are upgrading from 4.0 to 5.0 or 5.1 or from 4.1 to 5.1 please incrementally upgrade and then run a mysqlcheck -Agr during the upgrade to the next version''' (If old current MySQL version is pre-5.x, just do mysqlcheck -Ar)}}
 
  
If pre-MySQL 5.0:
+
APF SMTP tweak enables mail to be sent only from the mail or mailman GID, and blocks all outbound SMTP, except through the sendmail binary.
+
Add this '''bold''' line of code to /etc/init.d/apf , right underneath the start) case:
 
<pre>
 
<pre>
mysqlcheck -Aaor
+
/usr/local/sbin/apf --start >> /dev/null 2>&1
 +
'''/scripts/smtpmailgidonly on'''
 +
echo_success
 
</pre>
 
</pre>
  
If MySQL 5.0 to 5.1:
+
== FTP ==
 +
 
 +
If you are having issues with Proftp connections or with authentication. Check the Proftp configuration file below and make sure that "AuthPAM" is actually on.  
 +
<pre>
 +
vim /etc/proftpd.conf
 +
AuthPAM on
 +
</pre>
  
 +
If you want to make sure PureFTP  is using FTPES, edit /etc/pure-ftpd.conf and uncomment (enable) the PassivePortRange line, like below.
 
<pre>
 
<pre>
mysqlcheck -Agr
+
# Port range for passive connections replies. - for firewalling.
 +
PassivePortRange          30000 50000
 
</pre>
 
</pre>
  
And back up all the databases:
+
APF - /etc/apf/conf.apf
 +
<pre>
 +
# Common ingress (inbound) TCP ports
 +
IG_TCP_CPORTS="20,21,22,25,53,80,110,143,443,465,993,995,2082,2083,2084,2086,2087,2095,2096,3306,6666,7786,30000_50000"
  
<pre>
+
# Common egress (outbound) TCP ports
mkdir -p /backup/mysqldumps
+
EG_TCP_CPORTS="21,25,80,443,43,30000_50000"
cd /backup/mysqldumps
 
for i in $(mysql -e "show databases;" | cut -d ' ' -f2 | grep -v Database); do `mysqldump $i > $i.sql`; done
 
 
</pre>
 
</pre>
  
=== Enabling a Slow Query Log ===
 
  
How to enable a slow query log
+
CSF - /etc/csf/csf.conf
 +
<pre>
 +
# Allow incoming TCP ports
 +
TCP_IN = "20,21,22,25,53,80,110,143,443,465,953,993,995,2077,2078,2082,2083,2086,2087,2095,2096,30000:50000"
  
touch /var/lib/mysql/slow.log
+
# Allow outgoing TCP ports
+
TCP_OUT = "20,21,22,25,37,43,53,80,110,113,443,587,873,953,2087,2089,2703,30000:50000"
chown mysql. /var/lib/mysql/slow.log
+
</pre>
  
In the my.cnf file under the mysqld section add this:
 
  
log-slow-queries=/var/lib/mysql/slow.log
+
If you are encountering vsftp timeout issues or strange dns like issues with vsftp check the vsftpd configuration file and make sure that reverse_lookup_enable is set to no
 +
<pre>
 +
/etc/vsftpd/vsftpd.conf:
  
Then restart mysql and you have a slow query log.
+
reverse_lookup_enable=NO
 +
</pre>
  
If you want to specify the number of seconds that indicates a long or slow query, use this line in /etc/my.cnf :
+
== Nginx ==
  
long_query_time = 5
+
'''Common configuration settings'''
  
changing 5 to whatever number of seconds you want.
+
*The main configuration file to edit is /etc/nginx/nginx.conf, which by default also reaches out to include any additional configuration files in the conf.d directory and any virtual host files in the sites-enabled directory.
  
 +
*http://www.howtoforge.com/configuring-your-lemp-system-linux-nginx-mysql-php-fpm-for-maximum-performance
  
==PHP==
+
*'''worker_processes''' in /etc/nginx/nginx.conf. This should be equal to the amount of CPU cores the server has.
 +
<pre>
 +
worker_processes $CPUs;
 +
</pre>
  
===Parse Error===
+
*'''worker_connections''' defines how many connections each worker process is allowed to handle
  
Parse error: syntax error, unexpected T_STRING
+
*'''worker_processes x worker_connections''' tells the maximum amount of HTTP connections possible at any moment
  
Check the file and remove <?xml version="1.0" encoding="utf-8"?>
+
'''File cache settings'''
 +
<pre>
 +
http {
 +
[...]
 +
        ##
 +
        # File Cache Settings
 +
        ##
  
 +
        open_file_cache          max=5000  inactive=20s;
 +
        open_file_cache_valid    30s;
 +
        open_file_cache_min_uses 2;
 +
        open_file_cache_errors  on;
 +
</pre>
  
===Force PHP5===
+
'''Gzip''' This will compress content at the expense of a little extra CPU, but it will save a lot of bandwidth.
 +
<pre>
 +
gzip on;
 +
gzip_disable "msie6";
 +
gzip_min_length 1100;
 +
gzip_vary on;
 +
gzip_proxied any;
 +
gzip_buffers 16 8k;
 +
gzip_types text/plain text/css application/json application/x-javascript
 +
    text/xml application/xml application/rss+xml text/javascript
 +
    image/svg+xml application/x-font-ttf font/opentype
 +
    application/vnd.ms-fontobject;
 +
</pre>
  
Add to .htaccess:
+
'''Conflicting Server Name Error'''
  
AddType application/x-httpd-php5 .html .htm
+
Check for duplicates/system users:
 +
<pre>
 +
grep -i domain.com /var/cpanel/users/*
 +
</pre>
  
===Apache PHP Handlers===
+
If there is a domain entry owned by "system" remove this file:
 +
<pre>
 +
rm /var/cpanel/users/system
 +
</pre>
  
Can use this command to change owner and group
+
Then run:
 
<pre>
 
<pre>
chown -R user:group /directory/
+
/scripts/rebuildnginxvhost
 
</pre>
 
</pre>
  
===Install Zip from source===
+
== cPanel Tips and Tricks ==
  
 +
'''httpd.conf domain errors?'''
 
<pre>
 
<pre>
cd /usr/local/src
+
info [rebuildhttpdconf] Unable to determine group for $username, skipping domain $domain.com
wget http://pecl.php.net/get/zip-1.10.2.tgz
+
 
tar -zxvf zip-1.10.2.tgz
+
Check /var/cpanel/userdata/$user/$domain.com
cd zip-*
+
Make sure group: is set correctly
phpize
+
/scripts/rebuildhttpdconf
./configure
+
service httpd restart
make && make install
 
 
</pre>
 
</pre>
  
Install any extension from source:
+
Exclude files from being updated.
 
<pre>
 
<pre>
cd /usr/local/src
+
vim /etc/cpanelsync.exclude
wget somthing
 
tar -zxvf something.tgz
 
cd something-*
 
phpize
 
./configure
 
make && make install
 
echo "extension = something.so" >> /etc/php.ini
 
 
</pre>
 
</pre>
  
==DSO==
+
Then add the absolute path for the file. An example would be Roundcube webmail settings:
 +
 
 +
/usr/local/cpanel/base/3rdparty/roundcube/config/main.inc.php
 +
 
 +
'''Databases listed in Cpanel, but do not actually exist'''
  
Ownership - permissions should be 755
+
Check the following files and remove any users / dbs that do not exist:
 
<pre>
 
<pre>
user:nobody
+
/var/cpanel/databases/
 +
 
 +
$user.cache
 +
$user.yaml   
 
</pre>
 
</pre>
  
==FCGI==
+
'''spamd issues'''
 
 
Ownership - permissions should be 755
 
 
<pre>
 
<pre>
user:user
+
/scripts/perlinstaller IO::Socket::IP --force
 
</pre>
 
</pre>
  
==Email==
+
== DNS ==
  
 +
Disable zone transfers with named.conf
 +
<pre>
 +
acl can_axfr {
 +
127.0.0.1;
 +
};
  
===Email accounts not showing up in cPanel.===
+
options {
 +
    allow-recursion { trusted; };
 +
    allow-transfer { can_axfr; };
 +
};
 +
</pre>
  
Check /home/user/etc Make sure the passwd file and shadow file have proper permissions also make sure they are located in
+
'''WARNING: key file (/etc/rndc.key)'''
 
<pre>
 
<pre>
/home/user/etc/domain.com/
+
service named stop
 +
mv /etc/rndc.conf /etc/rndc.conf.OLD
 +
service named start
 
</pre>
 
</pre>
  
===Horde===  
+
== NFS ==
  
Can't find file: 'horde_sessionhandler.MYI'
 
 
<pre>
 
<pre>
/etc/init.d/mysqld stop
+
yum install nfs*
rm /var/lib/mysql/horde/horde_sessionhandler.frm
+
mkdir /$whatever/you/want/to/share
/etc/init.d/mysqld start
+
 
 +
vim /etc/exports
 +
added:
 +
/$whatever/you/want/to/share      $IPADDY/Subnetmask(rw,no_root_squash,subtree_check)
 +
 
 +
/etc/init.d/nfs start
 +
/etc/init.d/nfslock start
 +
/etc/init.d/rpcbind start
 +
/etc/init.d/rpcidmapd restart
 +
 
 +
vim /etc/idmapd.conf
 +
Uncommented / added:
 +
Domain = $local.domain.com
  
mysql
+
chkconfig rpcbind on
>
+
chkconfig rpcidmapd on
>CREATE TABLE horde_sessionhandler (session_id VARCHAR(32) NOT NULL, session_lastmodified INT NOT NULL, session_data LONGBLOB, PRIMARY KEY
+
chkconfig nfs on
(session_id)) ENGINE = InnoDB;
+
chkconfig nfslock on
  
>GRANT SELECT, INSERT, UPDATE, DELETE ON horde_sessionhandler TO [email protected];
+
Make sure port 2049 is open as well.
 
</pre>
 
</pre>
  
 +
== IPTABLES ==
  
 +
*http://wiki.centos.org/HowTos/Network/IPTables
  
===Spam===
+
This is an example of a default IPTABLES set of rules:
 +
<pre>
 +
*filter
 +
:INPUT ACCEPT [0:0]
 +
:FORWARD ACCEPT [0:0]
 +
:OUTPUT ACCEPT [0:0]
 +
:RH-Firewall-1-INPUT - [0:0]
 +
:TRUSTED - [0:0]
 +
-A INPUT -j RH-Firewall-1-INPUT
 +
-A FORWARD -j RH-Firewall-1-INPUT
 +
###Add trusted IPs / hosts / IP blocks here
 +
###Example would be:
 +
-A TRUSTED -s 192.168.0.0/24
 +
-A TRUSTED -s $myhomeIP
 +
-A TRUSTED -s $someotherserver
 +
###END TRUSTED HOSTS SECTION
 +
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
 +
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
 +
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
 +
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -j ACCEPT
 +
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
 +
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
 +
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 53 -j ACCEPT
 +
###EXAMPLE FOR ACTIVE/PASSIVE FTP ACCESS FOR TRUSTED HOSTS
 +
-A RH-Firewall-1-INPUT -p tcp --dport 21 -j TRUSTED
 +
-A RH-Firewall-1-INPUT -p tcp --dport 20 -j TRUSTED
 +
-A RH-Firewall-1-INPUT -p tcp --dport 30000:50000 -j TRUSTED
 +
###END FTP EXAMPLE
 +
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
 +
</pre>
  
Find top sending IPs in exim logs:
+
 
 +
 
 +
== Kernel Stuff ==
 +
 
 +
'''Tools and Utilities used to build a kernel'''
 
<pre>
 
<pre>
grep "SMTP connection from" /var/log/exim_mainlog |grep "connection count" |awk '{print $7}' |cut -d ":" -f 1 |cut -d "[" -f 2 |cut -d "]" -f 1 |sort -n |uniq -c | sort -n
+
gcc --version
 
</pre>
 
</pre>
 +
*Used to compile the kernel
  
Find authenticated users who may be spamming:
 
 
<pre>
 
<pre>
find /var/spool/exim/input/ -name '*-H' | xargs grep 'auth_id'
+
ld -v
 
</pre>
 
</pre>
 +
*Tools used to assist when compiling the kernel
  
Spam comming from scripts:
 
 
<pre>
 
<pre>
grep cwd=\/home\/ /var/log/exim_mainlog| cut -d' ' -f4 | sort | uniq -c | sort -n
+
make --version
 
</pre>
 
</pre>
 +
*Used to determine which files are needed to compile the kernel
  
Removing all queued messages at once in a safe way:
+
'''Tools and Utilities to use the kernel'''
 
<pre>
 
<pre>
exim -bp | awk '/^ *[0-9]+[mhd]/{print "exim -Mrm " $3}' | sh
+
fdformat --version
 
</pre>
 
</pre>
Or you can do the same from the mail queue manager in WHM.
+
*Used to handle mounting of disks
  
 +
<pre>
 +
depmod -V
 +
</pre>
 +
*Used to load kernel modules and remove them
  
APF SMTP tweak enables mail to be sent only from the mail or mailman GID, and blocks all outbound SMTP, except through the sendmail binary.
+
'''File System Tools'''
Add this '''bold''' line of code to /etc/init.d/apf , right underneath the start) case:
 
 
<pre>
 
<pre>
/usr/local/sbin/apf --start >> /dev/null 2>&1
+
tune2fs
'''/scripts/smtpmailgidonly on'''
 
echo_success
 
 
</pre>
 
</pre>
 +
*Used to handle the file systems such as ext4
  
===Relaying===
 
  
Add relaying from another server:
+
'''Command to see what modules are loaded:'''
 +
<pre>
 +
lsmod
 +
</pre>
  
Add the IP to the "remote service IPs" in cPanel
+
'''See all modules, even if they are not loaded:'''
 +
<pre>
 +
modprobe -l
 +
</pre>
  
===Sendmail===
+
'''Get detailed information on a module:'''
 +
<pre>
 +
modinfo $module
 +
</pre>
  
Find Spam in the queue:
+
'''Remove a module (assuming no other dependents are using it):'''
 
<pre>
 
<pre>
egrep -l "user" /var/spool/clientmqueue/Q* | wc -l
+
modprobe -r $module
 
</pre>
 
</pre>
  
==Packages==
+
'''See all kernel settings'''
 +
<pre>
 +
sysctl -a
 +
</pre>
  
To search for available packages:
+
'''TCP_FIN_TIMEOUT'''
 +
This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, making more resources available for new connections. Addjust this in the presense of many connections sitting in the TIME_WAIT state:
 
<pre>
 
<pre>
yum search example
+
# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
 +
(default: 60 seconds, recommended 15-30 seconds)
 
</pre>
 
</pre>
  
Find packages and where they lead to:
+
=== Steps to compile and customize a kernel ===
 +
The steps below will download the kernel source, decompress it, then will make the kernel with the default options.
 
<pre>
 
<pre>
rpm -qa | grep example
+
mkdir $place to put the kernel
 +
cd $place to put the kernel
 +
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.9.tar.xz
 +
xz -d linux-3.9.tar.xz
 +
tar -xvf linux-3.9.tar
 +
cd linux-3.9/
 +
make defconfig
 
</pre>
 
</pre>
 +
 +
From here, we can customize the kernel further.
 
<pre>
 
<pre>
rpm -ql example
+
make menuconfig
 
</pre>
 
</pre>
  
===Java + Tomcat===
+
Options when using menuconfig:
 
 
Regular install:
 
 
<pre>
 
<pre>
yum install java-1.6.0-openjdk.x86_64
+
[*] = Selected, if no star then not selected
yum install tomcat5
+
<Y> = Select module to be built into the Kernel
 +
<M> = Select module to be built as a module to be loaded, but not built into the kernel
 
</pre>
 
</pre>
  
==Apache==
+
== Postfix ==
  
'''Configuration file, lots of settings can be changed here:'''
+
Log location:
 
<pre>
 
<pre>
vim /usr/local/apache/conf/httpd.conf
+
/usr/local/psa/var/log/maillog
 
</pre>
 
</pre>
  
'''Includes (external settings that Apache reads in case the conf was rebuilt)'''
+
 
 +
Some one liners to figure out what is in the queue and how to remove bullshit emails.
 +
 
 
<pre>
 
<pre>
cd /usr/local/apache/conf/includes
+
mailq | grep ^[A-Z\|0-9] | awk '{print $7}' | cut [email protected] -f2 | sort | uniq -c | sort -rn | head -15
 
</pre>
 
</pre>
  
'''Check for a basic Dos, or heavy traffic:'''
+
Once you figure out senders or whatever, you can do something like this to either delete the email or put it in the hold queue
 +
 
 +
Put in hold queue
 
<pre>
 
<pre>
netstat -tn 2>/dev/null | grep ':80 ' | awk '{print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | head 
+
mailq | grep $someshittydomain.com | awk '{print $1}' | postsuper -h -
 
</pre>
 
</pre>
  
'''Count the processes:'''
+
Delete the emails
 
<pre>
 
<pre>
ps aux | grep httpd | wc -l
+
mailq | grep $someshittydomain.com | awk '{print $1}' |  postsuper -d -
ps aux | grep php | wc -l
 
 
</pre>
 
</pre>
  
 +
'''If these commands dont remove all the emails, you might need to use cut to get rid of the "!" or "*" which sometimes get placed at the end of the email id'''
 +
 +
== Benchmarking Tools ==
 +
 +
'''Please visit this page for more up to date information'''
 +
*http://wiki.mikejung.biz/index.php?title=Benchmarking
 +
 +
== ZFS ==
 +
'''This section is based off of an excellent guide by Ars.'''
 +
*http://arstechnica.com/information-technology/2014/02/ars-walkthrough-using-the-zfs-next-gen-filesystem-on-linux/
  
==OCFS2 and SAN==
 
  
cPanel not working for some accounts on some servers:
+
=== Creating ZFS Pool ===
  
 +
This will list available devices to use
 
<pre>
 
<pre>
chgrp user /var/cpanel/users/username
+
ls -l /dev/disk/by-id
vim /etc/proftpd/username
 
 
</pre>
 
</pre>
  
==FTP==
+
Once you determine what devices to use, this command will create the pool
 +
<pre>
 +
zpool create -o ashift=12 $name $raidz_type /dev/disk/by-id/$$ /dev/disk/by-id/$$ /dev/disk/by-id/$$
 +
</pre>
  
===PureFTP using FTPES===
+
'''NOTE'''
 +
* -o ashift=12 means "use 4K blocksizes instead of the default 512 byte blocksizes," which is appropriate on almost all modern drives.
  
Edit /etc/pure-ftpd.conf and uncomment (enable) the PassivePortRange line, like below.
+
=== ZFS Commands ===
 +
 
 +
This will display raw capacity status
 
<pre>
 
<pre>
# Port range for passive connections replies. - for firewalling.
+
zpool list
PassivePortRange          30000 50000
 
 
</pre>
 
</pre>
  
APF - /etc/apf/conf.apf
+
This will display usable status
 
<pre>
 
<pre>
# Common ingress (inbound) TCP ports
+
zfs list
IG_TCP_CPORTS="20,21,22,25,53,80,110,143,443,465,993,995,2082,2083,2084,2086,2087,2095,2096,3306,6666,7786,30000_50000"
+
</pre>
  
# Common egress (outbound) TCP ports
+
You can create "filesystems" which are much like pre-formated paritions or folders.
EG_TCP_CPORTS="21,25,80,443,43,30000_50000"
+
<pre>
 +
zfs create $zfs_vol/$folder_name
 
</pre>
 
</pre>
  
 +
You can and should create multiple filesystems so that you can manage each partition individually. If you have groups of content that you seperate already, then it makes sense to create multiple filesystems, such as images, movies, txt files, etc. By doing this you can take advantage of ZFS's settings.
  
CSF - /etc/csf/csf.conf
 
 
<pre>
 
<pre>
# Allow incoming TCP ports
+
zfs set compression=on $zfs_vol/textfiles
TCP_IN = "20,21,22,25,53,80,110,143,443,465,953,993,995,2077,2078,2082,2083,2086,2087,2095,2096,30000:50000"
+
zfs set quota=200G $zfs_vol/jpegs
 
 
# Allow outgoing TCP ports
 
TCP_OUT = "20,21,22,25,37,43,53,80,110,113,443,587,873,953,2087,2089,2703,30000:50000"
 
 
</pre>
 
</pre>
  
==Packages / Yum==
+
== View CPU Temps in Cent 6.5 ==
  
Install clamd on a coremanaged server:
+
For most new CPUs and Mobos this should be pretty simple to do. For this example, I'm using a newer SuperMicro Motherboard.
 
<pre>
 
<pre>
yum --disablerepo=\* --enablerepo=epel install clamd
+
## Install the package
</pre>
+
 
 +
yum -y install lm_sensors
  
 +
## Detect the sensors, should be fine to say YES to all the questions
  
==Load Balanced Troubleshooting==
+
sensors-detect
  
===Reboot Process===
+
## If everything installed correctly, you should see all the CPU core temps
  
Check to make sure SAN is mounted:
+
sensors
<pre>
 
netstat -lpn | grep 192.168
 
 
</pre>
 
</pre>
  
Stop OCFS2 before reboot:
+
 
 +
Example output, for this example I am using an Intel E5-1650v2
 
<pre>
 
<pre>
/etc/init.d/ocfs2 stop
+
coretemp-isa-0000
 +
Adapter: ISA adapter
 +
Physical id 0: +47.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 0:        +47.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 1:        +44.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 2:        +41.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 3:        +40.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 4:        +40.0°C  (high = +80.0°C, crit = +90.0°C) 
 +
Core 5:        +39.0°C  (high = +80.0°C, crit = +90.0°C) 
 
</pre>
 
</pre>
  
Start OCFS2:
+
== Linux Memory Usage Overview==
 +
 
 +
*http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html
 +
*http://stackoverflow.com/questions/7880784/what-is-rss-and-vsz-in-linux-memory-management
 +
 
 +
There are two commonly displayed values for Linux RAM usage. When using a tool like ps, you often times see VSZ and RSS.
 +
 
 +
'''VSZ''': "VSZ is the Virtual Memory Size. It includes all memory that the process can access, including memory that is swapped out and memory that is from shared libraries.
 +
"
 +
 
 +
'''RSS''': "RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.
 +
 
 +
*RSS And VSZ do not accurately represent the real RAM usage for a process, they report the total RAM the process would use if it were the only process running, but many processes share memory if they use the same shared libraries.
 +
 
 +
*Shared libraries like libc are commonly used by many different applications, Linux is able to load the library once into RAM, and then multiple processes can re-use the same library at the same time without having to duplicate the library which would use more RAM. Linux is very efficient because of its ability to share libraries among many processes.
 +
 
 +
 
 +
You can use pmap to get more specific memory usage information from a process.
 
<pre>
 
<pre>
/etc/init.d/ocfs2 start
+
pmap -d $PID
 
</pre>
 
</pre>
  
Make sure man is mounted:
+
An example command is:
 
<pre>
 
<pre>
mount | grep san
+
pmap -d 15441
 +
Address          Kbytes Mode  Offset          Device    Mapping
 +
....
 +
....
 +
00007f574e0a4000      8 rw--- 0000000000003000 0fc:00003 cStringIO.so
 +
00007f574e0a6000      20 r-x-- 0000000000000000 0fc:00003 stropmodule.so
 +
00007f574e0ab000    2044 ----- 0000000000005000 0fc:00003 stropmodule.so
 +
00007f574e2aa000      8 rw--- 0000000000004000 0fc:00003 stropmodule.so
 +
00007f574e2ac000      12 r-x-- 0000000000000000 0fc:00003 timemodule.so
 +
00007f574e2af000    2048 ----- 0000000000003000 0fc:00003 timemodule.so
 +
00007f574e4af000      8 rw--- 0000000000003000 0fc:00003 timemodule.so
 +
00007f5754477000    540 rw--- 0000000000000000 000:00000  [ anon ]
 +
00007f5754507000      12 rw--- 0000000000000000 000:00000  [ anon ]
 +
00007fff09ca1000    112 rw--- 0000000000000000 000:00000  [ stack ]
 +
00007fff09dff000      4 r-x-- 0000000000000000 000:00000  [ anon ]
 +
ffffffffff600000      4 r-x-- 0000000000000000 000:00000  [ anon ]
 +
mapped: 196340K    writeable/private: 9372K    shared: 0K
 
</pre>
 
</pre>
  
Restart Apache:
+
*The lines that have "r-x--" are considered the code segments.
 +
*The lines that have "rw---" are considered the data segments.
 +
 
 +
*The important information here is the "writeable/private" value, which is the incremental cost of the process once you remove all the other shared libraries that were already loaded / can be used by other processes.
 +
 
 +
Using an Apache process for another example:
 
<pre>
 
<pre>
service httpd restart
+
USER      PID %CPU %MEM    VSZ  RSS TTY      STAT START  TIME COMMAND
 +
nobody  22696  0.0  4.9 649624 49548 ?        Sl  17:45  0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
 
</pre>
 
</pre>
  
==Security==
+
*VSZ reports 649624K, or about 634MB
 +
*RSS reports 49548K, or about 48MB
  
Find all index.* files then remove bad things
+
Running pmap on that PID we see:
 
<pre>
 
<pre>
find /home/*/public_html/ -name index.* > /root/list
+
pmap -d 22696
</pre>
+
....
<pre>
+
....
for each in `cat /root/list` ; do sed -i.lwbak 's/Badthing\/script>//g' $each ; done
+
mapped: 649624K    writeable/private: 63292K    shared: 184140K
 
</pre>
 
</pre>
  
==Nginx==
+
*writeable/private: 63292K, or around 63MB, you can see that much of this process is using shared libraries.
 +
 
 +
==Storm and LiquidWeb API==
 +
 
 +
You can find API documentation at the link listed below.
  
===Conflicting Server Name Error===
+
*https://www.liquidweb.com/storm/api/docs/bleed/Storm/
  
Check for duplicates/system users:
+
If you have issues using the Liquid Web API the first step would be to run a simple curl command to make sure you can connect to the API and that are you using the correct user name and password. Please replace '''$API_USER''' and '''$API_PASS''' with your credentials. '''PLEASE be aware that this is not the most secure way to test this, you might want to throw this command into a file and run it that way, otherwise your credentials will be on the server's history, obviously this is not preferred.'''' You can create a temporary API user just to test, then remove the user or update the password.
 
<pre>
 
<pre>
grep -i domain.com /var/cpanel/users/*
+
curl https://$API_USER:[email protected].com/v1/utilities/info/ping.json
 
</pre>
 
</pre>
  
If there is a domain entry owned by "system" remove this file:
+
== Docker run command line examples ==
 +
 
 +
'''This command will run a container in interactive mode and will put you in the container as soon as it is started.'''
 
<pre>
 
<pre>
rm /var/cpanel/users/system
+
docker run -i -t -p $IP:$HostPort:$ContainerPort -v $HostDirectory:$ContainerDirectory $Image $Command
 
</pre>
 
</pre>
  
Then run:
+
An Example Command would be if you wanted to run a container with Apache that listens on port 80 in the container, and port 9000 on the host. We will also have the container use a directory on the host so that data persists even if the container is stopped or killed
 
<pre>
 
<pre>
/scripts/rebuildnginxvhost
+
docker run -p 8.8.8.8:9000:80 -v /partition1:/parition1 doge/apache:latest /usr/sbin/apache2ctl -D FOREGROUND
 
</pre>
 
</pre>
  
==Hardware Checks and Commands==
+
'''Quick and Dirty script to KILL off all containers'''
 
 
Check for disk age:
 
 
<pre>
 
<pre>
smartctl -a /dev/sda | grep Power_On_Hours
+
for each in `docker ps | awk '{print $1}'` ; do docker kill $each ; done
 
</pre>
 
</pre>
  
==cPanel Tips and Tricks==
+
'''Quick and Dirty script to STOP all containers, this is slower than the above command'''
 
 
Exclude files from being updated.
 
 
<pre>
 
<pre>
vim /etc/cpanelsync.exclude
+
for each in `docker ps | awk '{print $1}'` ; do docker stop $each ; done
 
</pre>
 
</pre>
  
Then add the absolute path for the file. An example would be Roundcube webmail settings:
+
==Linux Kernel Networking==
  
/usr/local/cpanel/base/3rdparty/roundcube/config/main.inc.php
+
A really good article that explains how networking performance in the Linux kernel will need some improvements in the near future. - https://lwn.net/Articles/629155/
 +
 +
[[Category:Wordpress]]
 +
[[Category:Optimization]]
 +
[[Category:Performance]]
 +
[[Category:Apache]]
 +
[[Category:PHP]]
 +
[[Category:Caching]]
 +
[[Category:MySQL]]
 +
[[Category:Linux]]
 +
[[Category:Ubuntu]]
 +
[[Category:CentOS]]
 +
[[Category:Database]]

Latest revision as of 15:12, 16 July 2019

Wiki Landing Page

Hello there! My name is Mike Jung and this is my wiki. I hope some of the information here will help someone out or teach them something new.

I realize you are likely here for technical content, which is exactly why you should check out these scanned vintage Real photo postcards from the 1900s, some of them are kinda cool actually!


Wiki Navigation and Page List

Collectible Trading Cards

This section covers various collectable trading cards from the 1990s and 2000s as well as collectable trading card games such as Pokemon, Yu-Gi-Oh and Magic the Gathering. Many of these artists have done work for major comic book companies, movies, tv shows and other forms of media that have enhanced our lives over the years.

  • Mike Ploog - Mike Ploog is a famous comic artist that has worked on countless movies, comics, tv shows and other works.
  • Tim and Greg Hildebrandt - These two brothers have created Star Wars movie posters, drawn countless X-Men, worked on LOTR and countless other epic things, check out their work!
  • 1993 SkyBox DC Cosmic Teams - One of my favorite trading card series from DC / SkyBox from the early 1990s, check out the cool holograms!

Hearthstone

  • Legendary Ramp Druid - If you like Hearthstone, Legendary cards, and playing a Druid, visit this page!

Windows 8.1 Utilities and Windows Tech Preview 10

  • MPC-HC Audio Configuration - While this page focuses on configuring MPC-HC for the best audio quality, it can still be used to configure Windows 8.1 audio for high end sound cards and speakers such as the Logitech Z 906 speakers and the Sound Blaster Z sound card.
  • Windows 10 Tech Preview - Currently testing out Windows 10 Tech Preview, Build 9926. This wiki page will eventually have tons of information and updates on Windows 10.
  • CrystalDiskMark - A beginner guide on how to use CrystalDiskMark to benchmark your HDD or SSD.
  • S3 Browser - S3 Browser is similar to an FTP client, but it speaks to a REST endpoint of an S3 compatible Object Storage service. AWS S3 is supported, but other compatible Object Storage services are also supported.
  • MSI Afterburner - MSI Afterburner configuration guide / walk through with screenshots that explain how to do stuff with afterburner, like select GPU usage in OSD.
  • Virtualbox - Various tips and tricks relating to VirtualBox, which allows you to run virtual machines on Windows hosts.

GPU and Video Decoding Stuff (madVR, CUDA, MPC-HC)

  • Geekbench4 - How to use GeekBench4 to measure CPU and GPU compute performance.
  • Nvidia Noise Reduction - Nvidia Noise Reduction reduces noise during video playback, how much does this improve quality? Not a ton, in some cases, anyway, check out the page for more info.
  • Nvidia Edge Enhancement - Ever wonder if the Nvidia Video settings make much of a difference in terms of video quality? Turns out it can make a huge difference! This page focuses on Nvidia Edge Enhancements
  • MPC-HC Video Decoder Comparison - Screenshot comparisons between MPC-HC's default Video Decoder (DXVA) and CUVID. If you have a Nvidia GPU and want to make sure you have the best video quality, check out this page!
  • DyingLight - Some PC screenshots of DyingLight, using Nvidia DSR with a GTX 970. DSR does improve quality, but is very costly in terms of performance.
  • Windows 8.1 MPC-HC and MadVR Setup Guide - A Windows 8.1 based guide on how to properly configure MPC-HC (media player classic home cinema) to work with MadVR as the video renderer. This setup guide includes images for each of the steps and explanations of the main mpc-hc and madvr settings.
  • CUDA - GPUs are on track to control the global population. CUDA is how the matrix started. It's pretty cool though, CUDA allows a GPU to accelerate some types of processing that previously only the CPU could compute.
  • DXVA2 - A quick article that explains what DXVA2 is and how it interacts with a GPU during the video playback process.
  • MadVR - MadVR MPC-HC wiki containing optimization tips and benchmarks for Chroma Upscaling, Image Doubling, Image Upscaling, Image downscaling and many other configuration settings.
  • PotPlayer - PotPlayer wiki about how to install, configure and optimize potplayer using MadVR, CUDA and GPU magic
  • DirectShow - What is directshow? What does directshow do? Want to learn more about directshow? Then please visit this page!
  • MadVR Chroma Upscaling - MadVR Chroma Upscaling performance results and general information on the best scaling algorithm to use to upscale Chroma with MadVR.
  • Nnedi3 - madVR recently added the ability to use NNEDI3 to improve video quality, this wiki covers how to configure this.

cPanel Stuff

  • CloudLinux -- Overview on what CloudLinux is and the types of resources that it limits for cpanel users.
  • PHP Handlers - How to change cPanel's main PHP handler via CLI and via WHM.
  • LiquidWeb WHMCS Plugin - A FREE WHMCS plugin created by ModulesGarden that allows anyone to resell SSD VPS servers, or whatever kind of servers you want. Basically this plugin is a Cloud Business in a box and it's free.
  • CPanel Statistics Software - Learn how to configure cPanel to process logs and bandwidth during low traffic hours of the day. This reduces the amount of Apache restarts.
  • Whmcs 6 - Need help installing and configuring WHMCS 6? Visit this page!
  • Ioncube loader - Are you getting ioncube loader errors or encountering version issues? Visit my ioncube loader page to learn more!

Webserver Stuff

  • Litespeed - Information about the litespeed webserver installation process and how to correctly configure litespeed on a cpanel server.
  • Apache - Do you like websites? You can thank Apache! It's the most common webserver around. Nginx is gaining some steam, but Apache is still pretty awesome!
  • GooglePageSpeed - How to install and configure the mod_pagespeed module with Apache, in addition, and really importantly, How to install Google's mod_pagespeed on a cPanel server via Easy Apache. lol
  • WordPress Optimization - Guide on how to optimize a server for wordpress. Includes Apache Event configuration, PHP-FPM configuration, MySQL configuration and lots more!

MySQL, PHP and Caching

  • PHP 7 - This wiki page is still being worked on, but it will focus specifically on PHP 7 and how awesome it is.
  • PHP MySQL Extensions - Also still in the works, will focus specifically on the MySQL extensions for PHP.
  • Memcached - Caching makes everything faster! I like fast things, so I use memcached a lot and you should use memcached too! I'll show you how to use memcached to improve website load time and reduce latency when connecting to a database! All of this can be done if you know how to tame the mythical beast know as memcached.
  • PHP_OPcache - Do you like fast things? Want to make PHP faster? Use opcode caching. For your health!
  • fcgid - FastCGI will make ur blog faster! Maybe, if you know how to configure Apache to use FastCGI to proxy PHP requests to a dedicated PHP process! If you want to learn more about using the FCGI handler on cpanel, please check out this page!
  • Php-fpm - Speaking of awesome...PHP-FPM is here. Are you still using mod_php and wondering why apche is slow? It's because you are doing php wrong! Check out this page for information on how to install, tune, and optimize php-fpm with apache
  • Varnish - Varnish can be an excellent way to improve your website's response times by caching the most used static content, like images, varnish is able to significantly reduce the amount of work that apache has to do.
  • MySQL - Learn all about the various my.cnf settings for MySQL 5.5 and MySQL 5.6. This page is less focused on optimizations, and more focused on covering the best practices for configuring a server to handle MySQL.
  • MediaWiki Performance Tweaks - I use MediaWiki, if you can't tell already. I've created a page that lists all the caching settings / configurations to make this site somewhat fast.
  • HHVM - Facebook's attempt to take PHP to the next level. HHVM uses a JIT compiler to significantly reduce the amount of resources it takes to interpret and run a PHP script.

Monitoring and Analysis

  • Newrelic - Newrelic is pretty awesome. They offer a free tier which lets you monitor server resources for 24 hours. You can also utilize APM which is an application monitoring services which shows the response time if your application and database. If you are looking for common Newrelic agent commands or need help troubleshooting Newrelic's agents, check out this page!
  • Sysdig - Looking for a utility that will provide insight into application and Linux performance? Sysdig is your tool! I really like it and find it pretty useful so I made a wiki!
  • Sysstat - Sysstat contains sar which is used to record server resource usage over the course of each day. Sar is really helpful if you care about server performance so knowing how to view data like swap in and swap out activity is critical.

Browser and Front End

  • Chrome - A list of tweaks (flags) that you can enable in the Chrome and Chromium web browsers which can help to speed up performance. Useful if you notice slow, laggy websites and want to speed up your browser.
  • HTTP 2.0 - Still in creation mode, this wiki will eventually contain all kinds of information on the new HTTP 2.0 protocol.
  • Browser Caching - .htaccess examples for how to enable browser caching of images and static files.

Benchmarking and Performance Tuning Stuff

  • Benchmarking -- A linux benchmarking reference wiki with many example commands and explanations for sysbench, fio, iozone and ioping tests.
  • Sysbench -- Similar to the benchmarking wiki but with 100% focus on sysbench and how to benchmark vps and cloud servers.
  • Phoronix Test Suite -- How to install Phoronix Test Suite on CentOS and Ubuntu. Examples on how to use the test suite and an overview on what each test does.
  • OS Tuning - You can't tune an application until you tune the operating system. Check out my OS system tuning wiki for tips and tricks on speeding up your slow CentOS server.
  • Dmcache - Caching, SSDs, what could be better? What about using SSDs to cache your slow as balls hdds? Learn how to by checking out this dmcache wiki!
  • Siege - Learn how to compile siege so you can test website performance on http and https websites

Storage and File System Stuff

  • LVM Commands - LVM command reference guide. Explains what logical volumes and logical groups are all about and how to create an LVM volume
  • Ceph - Ceph is a distributed storage system that powers the open cloud and internet of things. Just kidding, it doesn't do all that but it is still pretty awesome technology!
  • LSI - LSI makes RAID cards. Been around for a long time, recently bought out by seagate. LSI cards are nice, but sometimes slow if you do not configure RAID for performance. If you want to add some performance to your RAID, make sure you configured the card correctly!
  • DRBD - Data replicating block device, aka DRBD has been a heavyweight in the cloud storage wars for a while now. You got Ceph in one corner, OCFS2 in another, RAID (for backups) and DRBD. DRBD can be tricky to configure and even if you get it to work it might still be somewhat slow. I have created a wiki that covers some basic performance tuning for DRBD.
  • Big Data - Main page that links to topics like Cassandra and Hadoop.

Linux Kernel Stuff

  • Software RAID - How to use mdadm and configure software RAID on Linux
  • Ubuntu - General wiki for how to do things on Ubuntu.

Other Stuff

  • Firewalld - Are you used to iptables and hate firewalld? It's ok, I feel your pain! Firewalld is actually pretty easy to use so if you're running CentOS 7 or Redhat 7 please visit this page.
  • Hearthstone Combo Guide - I'm a Hearthstone nub, so I made a wiki on how to build a deck that will at least give you some options when it comes to minion combinations.
  • Real photo postcards - Totally random old picture post card wiki. Completely unrelated to anything that has to do with the internet.
  • CentOS 7 - Main page for CentOS 7 which includes all kinds of performance tweaks for the 3.10 kernel or newer.
  • Load Balancing - Learn more about the Stingray / Riverbed Traffic Manager! It's pretty cool and has a ton of options, if you are looking for some load balancing information, check out the wiki!
  • How to save bash directory location - Bashmarks Is a simple tool that allows you to save directory locations and then later return to them using extremely simple commands that even tab complete!
  • Cassandra - Cassandra is a NOSQL like DB that Apache made. This wiki contains general information about what Cassandra is, how it works and details on the topology.
  • Hadoop - Also NoSQL like, hadoop is great for running batch jobs against a large amount of data.
  • Gcc CentOS - Why is GCC always old on CentOS? Why does CentOS always ship old software? I do not know, but I can show you how to update GCC on CentOS if you visit the GCC CENTOS wiki!
  • Hardware - Wiki that covers SSDs, PCIe Flash SSDs, RAID cards and other types of server hardware. Includes tips and tweaks to improve performance and stability.
  • Security - Basic ways to secure your server and website. How to track down malicious files and how to clean up a hacked website.
  • R1soft - Idera / R1soft backup manager overview
  • Processor - Basic information about CPUs and how they work
  • ApacheTheory - Information about how Apache works, how to optimize Apache and an explanation on the role of Apache.
  • Logs - A list of where the common cPanel log locations are and what type of information they hold.
  • Sysstat - sysstat includes sar, which is awesome

for loop example script

Sed

Add a word to the begining of a line

sed 's/^/$Wordtoadd/' original.txt > sorted_original.txt

Linux commands to check for DDoS and excessive connections

This section is being merged into the main Apache wiki which can be found here

Web Permissions | Files

Default Web Permissions NOTE: MAKE SURE YOU ARE IN A public_html directory!!!!!!

find . -type f -exec chmod 644 {} \;
find . -type d -exec chmod 755 {} \;


Find all users php.ini files.

find /home/*/public_html/* -name php.ini

Modules / Directives

speling

mod_speling.c

Once added via Easy Apache, you can simply add these directives to a .htaccess file

CheckCaseOnly On
CheckSpelling On

How To Optimize WordPress

For a detailed guide, please visit my [WordPress Optimization Guide]

PHP

Install ssh2 Pecl extension

yum install libssh2 libssh2-devel
pecl install ssh2
# You may need to update the channel, if so;
pecl channel-update pecl.php.net

Then just;
vim /etc/php.ini
extension=ssh2.so

Upload issues

May need to check two files, the first is the global php.ini file, the next is the modsec file (if applicable)

vim /usr/local/lib/php.ini

upload_tmp_dir = /tmp
session.save_path = /tmp

----------------------------------------------

vim /usr/local/apache/conf/modsec2/custom.conf

SecUploadDir /tmp
SecTmpDir /tmp

Parse Error

Parse error: syntax error, unexpected T_STRING

Check the file and remove <?xml version="1.0" encoding="utf-8"?>


Force PHP5

Add to .htaccess:

AddType application/x-httpd-php5 .html .htm

Email

How to enable DKIM for a cpanel account

  • DomainKeys Identified Mail (DKIM) defines a mechanism by which email messages can be cryptographically signed, permitting a signing domain to claim responsibility for the introduction of a message into the mail stream. Message recipients can verify the signature by querying the signer's domain directly to retrieve the appropriate public key, and thereby confirm that the message was attested to by a party in possession of the private key for the signing domain.
  • To verify that everything is setup correctly you can send an email from an email account on that domain to [email protected] No need to have a subject or body. This service will then reply with a message stating the verification of DKIM, DomainKeys, SPF, SpamAssassin, and Sender-ID. Great tool to test all kinds of email verification systems.

To install on a cPanel server:

/usr/local/cpanel/bin/dkim_keys_install <username>

or

for i in `ls /var/cpanel/users`; do /usr/local/cpanel/bin/dkim_keys_install $i; done
  • Add the Policy Record
_domainkey     IN     TXT     "t=y; o=~; n=Interim Sending Domain Policy; [email protected]"

General webmail and email permission guidelines for cPanel servers

Below are some baseline permissions that should be used with Exim and Dovecot:

/home/user/etc/

domain.com file should have:
permissions:
750
ownership:
username:mail

/home/user/etc/domain.com/

passwd
permissions:
640
ownership:
user:mail

quota
permissions:
640
ownership:
user:mail

shadow
permissions:
640
ownership:
user:user

/home/user/mail/

700 user:user cur/
751 user:user domain.com/
700 user:user anything else

If email accounts are not showing up in cPanel for a specific cpanel user be sure to check /home/$user/etc to make sure the passwd file and shadow file have proper permissions also make sure they are located in

/home/user/etc/domain.com/

If all the permissions are correct and the directories are owned by the user, try restarting cpanel mail services to see if this helps resolve the issue.


If you run into a Roundcube error like "unable to connect to database", the best thing to do is to drop the database, then re-install roundcube, which automatically re-creates the db. Make sure you backup the database before you drop it, or else you risk lots of possible data loss

cd /home/temp 
mysqldump roundcube > roundcube.sql 
mysql -e "drop database roundcube;" 
/usr/local/cpanel/bin/update-roundcube --force


If you are running into spam issues you can run the command below to find top sending IPs in exim logs:

grep "SMTP connection from" /var/log/exim_mainlog |grep "connection count" |awk '{print $7}' |cut -d ":" -f 1 |cut -d "[" -f 2 |cut -d "]" -f 1 |sort -n |uniq -c | sort -n

Find authenticated users who may be spamming:

find /var/spool/exim/input/ -name '*-H' | xargs grep 'auth_id'

Spam comming from scripts:

grep cwd=\/home\/ /var/log/exim_mainlog| cut -d' ' -f4 | sort | uniq -c | sort -n

Removing all queued messages at once in a safe way:

exim -bp | awk '/^ *[0-9]+[mhd]/{print "exim -Mrm " $3}' | sh

Or you can do the same from the mail queue manager in WHM.


APF SMTP tweak enables mail to be sent only from the mail or mailman GID, and blocks all outbound SMTP, except through the sendmail binary. Add this bold line of code to /etc/init.d/apf , right underneath the start) case:

/usr/local/sbin/apf --start >> /dev/null 2>&1
'''/scripts/smtpmailgidonly on'''
echo_success

FTP

If you are having issues with Proftp connections or with authentication. Check the Proftp configuration file below and make sure that "AuthPAM" is actually on.

vim /etc/proftpd.conf
AuthPAM on

If you want to make sure PureFTP is using FTPES, edit /etc/pure-ftpd.conf and uncomment (enable) the PassivePortRange line, like below.

# Port range for passive connections replies. - for firewalling.
PassivePortRange          30000 50000

APF - /etc/apf/conf.apf

# Common ingress (inbound) TCP ports
IG_TCP_CPORTS="20,21,22,25,53,80,110,143,443,465,993,995,2082,2083,2084,2086,2087,2095,2096,3306,6666,7786,30000_50000"

# Common egress (outbound) TCP ports
EG_TCP_CPORTS="21,25,80,443,43,30000_50000"


CSF - /etc/csf/csf.conf

# Allow incoming TCP ports
TCP_IN = "20,21,22,25,53,80,110,143,443,465,953,993,995,2077,2078,2082,2083,2086,2087,2095,2096,30000:50000"

# Allow outgoing TCP ports
TCP_OUT = "20,21,22,25,37,43,53,80,110,113,443,587,873,953,2087,2089,2703,30000:50000"


If you are encountering vsftp timeout issues or strange dns like issues with vsftp check the vsftpd configuration file and make sure that reverse_lookup_enable is set to no

/etc/vsftpd/vsftpd.conf:

reverse_lookup_enable=NO

Nginx

Common configuration settings

  • The main configuration file to edit is /etc/nginx/nginx.conf, which by default also reaches out to include any additional configuration files in the conf.d directory and any virtual host files in the sites-enabled directory.
  • worker_processes in /etc/nginx/nginx.conf. This should be equal to the amount of CPU cores the server has.
worker_processes $CPUs;
  • worker_connections defines how many connections each worker process is allowed to handle
  • worker_processes x worker_connections tells the maximum amount of HTTP connections possible at any moment

File cache settings

http {
[...]
        ##
        # File Cache Settings
        ##

        open_file_cache          max=5000  inactive=20s;
        open_file_cache_valid    30s;
        open_file_cache_min_uses 2;
        open_file_cache_errors   on;

Gzip This will compress content at the expense of a little extra CPU, but it will save a lot of bandwidth.

gzip on;
gzip_disable "msie6";
gzip_min_length 1100;
gzip_vary on;
gzip_proxied any;
gzip_buffers 16 8k;
gzip_types text/plain text/css application/json application/x-javascript
    text/xml application/xml application/rss+xml text/javascript
    image/svg+xml application/x-font-ttf font/opentype
    application/vnd.ms-fontobject;

Conflicting Server Name Error

Check for duplicates/system users:

grep -i domain.com /var/cpanel/users/*

If there is a domain entry owned by "system" remove this file:

rm /var/cpanel/users/system

Then run:

/scripts/rebuildnginxvhost

cPanel Tips and Tricks

httpd.conf domain errors?

info [rebuildhttpdconf] Unable to determine group for $username, skipping domain $domain.com

Check /var/cpanel/userdata/$user/$domain.com
Make sure group: is set correctly
/scripts/rebuildhttpdconf
service httpd restart

Exclude files from being updated.

vim /etc/cpanelsync.exclude

Then add the absolute path for the file. An example would be Roundcube webmail settings:

/usr/local/cpanel/base/3rdparty/roundcube/config/main.inc.php

Databases listed in Cpanel, but do not actually exist

Check the following files and remove any users / dbs that do not exist:

/var/cpanel/databases/

$user.cache
$user.yaml    

spamd issues

/scripts/perlinstaller IO::Socket::IP --force

DNS

Disable zone transfers with named.conf

acl can_axfr {
127.0.0.1;
};

options {
    allow-recursion { trusted; };
    allow-transfer { can_axfr; };
};

WARNING: key file (/etc/rndc.key)

service named stop
mv /etc/rndc.conf /etc/rndc.conf.OLD
service named start

NFS

yum install nfs*
mkdir /$whatever/you/want/to/share

vim /etc/exports
added:
/$whatever/you/want/to/share       $IPADDY/Subnetmask(rw,no_root_squash,subtree_check)

/etc/init.d/nfs start
/etc/init.d/nfslock start
/etc/init.d/rpcbind start
/etc/init.d/rpcidmapd restart

vim /etc/idmapd.conf
Uncommented / added:
Domain = $local.domain.com

chkconfig rpcbind on
chkconfig rpcidmapd on
chkconfig nfs on
chkconfig nfslock on

Make sure port 2049 is open as well. 

IPTABLES

This is an example of a default IPTABLES set of rules:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
:TRUSTED - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
###Add trusted IPs / hosts / IP blocks here
###Example would be:
-A TRUSTED -s 192.168.0.0/24
-A TRUSTED -s $myhomeIP
-A TRUSTED -s $someotherserver
###END TRUSTED HOSTS SECTION
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 53 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 53 -j ACCEPT
###EXAMPLE FOR ACTIVE/PASSIVE FTP ACCESS FOR TRUSTED HOSTS
-A RH-Firewall-1-INPUT -p tcp --dport 21 -j TRUSTED
-A RH-Firewall-1-INPUT -p tcp --dport 20 -j TRUSTED
-A RH-Firewall-1-INPUT -p tcp --dport 30000:50000 -j TRUSTED
###END FTP EXAMPLE
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited


Kernel Stuff

Tools and Utilities used to build a kernel

gcc --version
  • Used to compile the kernel
ld -v
  • Tools used to assist when compiling the kernel
make --version
  • Used to determine which files are needed to compile the kernel

Tools and Utilities to use the kernel

fdformat --version
  • Used to handle mounting of disks
depmod -V
  • Used to load kernel modules and remove them

File System Tools

tune2fs
  • Used to handle the file systems such as ext4


Command to see what modules are loaded:

lsmod

See all modules, even if they are not loaded:

modprobe -l 

Get detailed information on a module:

modinfo $module

Remove a module (assuming no other dependents are using it):

modprobe -r $module 

See all kernel settings

sysctl -a

TCP_FIN_TIMEOUT This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, making more resources available for new connections. Addjust this in the presense of many connections sitting in the TIME_WAIT state:

# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
(default: 60 seconds, recommended 15-30 seconds)

Steps to compile and customize a kernel

The steps below will download the kernel source, decompress it, then will make the kernel with the default options.

mkdir $place to put the kernel
cd $place to put the kernel
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.9.tar.xz
xz -d linux-3.9.tar.xz
tar -xvf linux-3.9.tar
cd linux-3.9/
make defconfig

From here, we can customize the kernel further.

make menuconfig

Options when using menuconfig:

[*] = Selected, if no star then not selected
<Y> = Select module to be built into the Kernel
<M> = Select module to be built as a module to be loaded, but not built into the kernel

Postfix

Log location:

/usr/local/psa/var/log/maillog


Some one liners to figure out what is in the queue and how to remove bullshit emails.

mailq | grep ^[A-Z\|0-9] | awk '{print $7}' | cut [email protected] -f2 | sort | uniq -c | sort -rn | head -15

Once you figure out senders or whatever, you can do something like this to either delete the email or put it in the hold queue

Put in hold queue

mailq | grep $someshittydomain.com | awk '{print $1}' |  postsuper -h -

Delete the emails

mailq | grep $someshittydomain.com | awk '{print $1}' |  postsuper -d -

If these commands dont remove all the emails, you might need to use cut to get rid of the "!" or "*" which sometimes get placed at the end of the email id

Benchmarking Tools

Please visit this page for more up to date information

ZFS

This section is based off of an excellent guide by Ars.


Creating ZFS Pool

This will list available devices to use

ls -l /dev/disk/by-id

Once you determine what devices to use, this command will create the pool

zpool create -o ashift=12 $name $raidz_type /dev/disk/by-id/$$ /dev/disk/by-id/$$ /dev/disk/by-id/$$

NOTE

  • -o ashift=12 means "use 4K blocksizes instead of the default 512 byte blocksizes," which is appropriate on almost all modern drives.

ZFS Commands

This will display raw capacity status

zpool list

This will display usable status

zfs list

You can create "filesystems" which are much like pre-formated paritions or folders.

zfs create $zfs_vol/$folder_name

You can and should create multiple filesystems so that you can manage each partition individually. If you have groups of content that you seperate already, then it makes sense to create multiple filesystems, such as images, movies, txt files, etc. By doing this you can take advantage of ZFS's settings.

zfs set compression=on $zfs_vol/textfiles
zfs set quota=200G $zfs_vol/jpegs

View CPU Temps in Cent 6.5

For most new CPUs and Mobos this should be pretty simple to do. For this example, I'm using a newer SuperMicro Motherboard.

## Install the package

yum -y install lm_sensors

## Detect the sensors, should be fine to say YES to all the questions

sensors-detect

## If everything installed correctly, you should see all the CPU core temps

sensors


Example output, for this example I am using an Intel E5-1650v2

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +47.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 0:        +47.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 1:        +44.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 2:        +41.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 3:        +40.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 4:        +40.0°C  (high = +80.0°C, crit = +90.0°C)  
Core 5:        +39.0°C  (high = +80.0°C, crit = +90.0°C)  

Linux Memory Usage Overview

There are two commonly displayed values for Linux RAM usage. When using a tool like ps, you often times see VSZ and RSS.

VSZ: "VSZ is the Virtual Memory Size. It includes all memory that the process can access, including memory that is swapped out and memory that is from shared libraries. "

RSS: "RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.

  • RSS And VSZ do not accurately represent the real RAM usage for a process, they report the total RAM the process would use if it were the only process running, but many processes share memory if they use the same shared libraries.
  • Shared libraries like libc are commonly used by many different applications, Linux is able to load the library once into RAM, and then multiple processes can re-use the same library at the same time without having to duplicate the library which would use more RAM. Linux is very efficient because of its ability to share libraries among many processes.


You can use pmap to get more specific memory usage information from a process.

pmap -d $PID

An example command is:

pmap -d 15441
Address           Kbytes Mode  Offset           Device    Mapping
....
....
00007f574e0a4000       8 rw--- 0000000000003000 0fc:00003 cStringIO.so
00007f574e0a6000      20 r-x-- 0000000000000000 0fc:00003 stropmodule.so
00007f574e0ab000    2044 ----- 0000000000005000 0fc:00003 stropmodule.so
00007f574e2aa000       8 rw--- 0000000000004000 0fc:00003 stropmodule.so
00007f574e2ac000      12 r-x-- 0000000000000000 0fc:00003 timemodule.so
00007f574e2af000    2048 ----- 0000000000003000 0fc:00003 timemodule.so
00007f574e4af000       8 rw--- 0000000000003000 0fc:00003 timemodule.so
00007f5754477000     540 rw--- 0000000000000000 000:00000   [ anon ]
00007f5754507000      12 rw--- 0000000000000000 000:00000   [ anon ]
00007fff09ca1000     112 rw--- 0000000000000000 000:00000   [ stack ]
00007fff09dff000       4 r-x-- 0000000000000000 000:00000   [ anon ]
ffffffffff600000       4 r-x-- 0000000000000000 000:00000   [ anon ]
mapped: 196340K    writeable/private: 9372K    shared: 0K
  • The lines that have "r-x--" are considered the code segments.
  • The lines that have "rw---" are considered the data segments.
  • The important information here is the "writeable/private" value, which is the incremental cost of the process once you remove all the other shared libraries that were already loaded / can be used by other processes.

Using an Apache process for another example:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
nobody   22696  0.0  4.9 649624 49548 ?        Sl   17:45   0:01  \_ /usr/local/apache/bin/httpd -k start -DSSL
  • VSZ reports 649624K, or about 634MB
  • RSS reports 49548K, or about 48MB

Running pmap on that PID we see:

pmap -d 22696
....
....
mapped: 649624K    writeable/private: 63292K    shared: 184140K
  • writeable/private: 63292K, or around 63MB, you can see that much of this process is using shared libraries.

Storm and LiquidWeb API

You can find API documentation at the link listed below.

If you have issues using the Liquid Web API the first step would be to run a simple curl command to make sure you can connect to the API and that are you using the correct user name and password. Please replace $API_USER and $API_PASS with your credentials. PLEASE be aware that this is not the most secure way to test this, you might want to throw this command into a file and run it that way, otherwise your credentials will be on the server's history, obviously this is not preferred.' You can create a temporary API user just to test, then remove the user or update the password.

curl https://$API_USER:[email protected]/v1/utilities/info/ping.json

Docker run command line examples

This command will run a container in interactive mode and will put you in the container as soon as it is started.

docker run -i -t -p $IP:$HostPort:$ContainerPort -v $HostDirectory:$ContainerDirectory $Image $Command

An Example Command would be if you wanted to run a container with Apache that listens on port 80 in the container, and port 9000 on the host. We will also have the container use a directory on the host so that data persists even if the container is stopped or killed

docker run -p 8.8.8.8:9000:80 -v /partition1:/parition1 doge/apache:latest /usr/sbin/apache2ctl -D FOREGROUND

Quick and Dirty script to KILL off all containers

for each in `docker ps | awk '{print $1}'` ; do docker kill $each ; done

Quick and Dirty script to STOP all containers, this is slower than the above command

for each in `docker ps | awk '{print $1}'` ; do docker stop $each ; done

Linux Kernel Networking

A really good article that explains how networking performance in the Linux kernel will need some improvements in the near future. - https://lwn.net/Articles/629155/