(转)Magento资源

Standard

http://www.fromdev.com/2014/07/best-magento-tutorials-ebooks-pdf.html

50+ Best Magento Tutorials, eBooks and PDF Resources

Magento Tutorials, eBooks and PDF Resources

Magento has already made some massive strides in the web development landscape and going by the rate with which it is being adopted by the web development professionals and businesses, this immensely resourceful content management system is poised for even higher goals.

Now, as web development companies and merchants are increasingly leaning towards Magento to build their online stores, those with aspirations to step into the big and bang world of web development are seriously considering gaining expertise in Magento and using it as a stepping stone for their career – before they can contribute to this platform’s further growth and advancement.

Now, you may or may not find a Magento training institute in your neighborhood, but there are host of ebooks, PDFs and tutorials to help you in your cause. Beginners, specifically, must use more than one books as a resource for catching the Magento wagon. And if you are someone with not a lot of coding experience, but with a drive to build your own Ecommerce website, the following magento tutorial will serve the purpose just right:

Best Magento Tutorials For Beginners

The Official Magento User Guide

Not just those striving to be Magento developers, but even the store owners and merchants looking to enhance their eCommerce website can also resort to this tutorial which serves to be the comprehensive guide for gaining a deeper insight into the nitty-gritty of the Magento CMS and how it can be leveraged for sprucing up the online stores. Whether you wish to:

  • Set up the hierarchy on your site
  • Arrange the products in the form of catalogs and menus
  • Facilitate a more streamlined navigational structure
  • or accomplish any other store-specific task, this tutorial will guide you achieve so it in a manner most potent and accommodating.

Magento: Beginners Guide

This is another tool that takes you to the very basics of Magento and starts doling out some serious and relevant sermon so that you don’t skimp on the most crucial concepts. When setting up an online store is on your agenda, this step by step guide would prove to be a priceless asset for you to get a grasp on the most underlying and crucial concepts required to do so. The guide also describes how you can customize your Magento store easily.

Magento: User Guide

A remarkably resourceful guide for the novices in web development, this tutorial helps by:
Giving tips on how to set up a web store without having to code
Introduces you to the most basic and most crucial concepts that facilitate fuss-free Magento development

For Those Striving for Advanced Magento Expertise
Well, you never stop learning, and the following tutorials are a testament to it:

Designer’s Guide To Magento (PDF)

If you are someone with hands-on experience and a reasonably impressive level of skills in Magento, then this guide is what you need to further broaden your horizon and hone your skills. The guide is updated for the latest versions like CEv1.4+, PEv1.8+ and EEv1.8+. While you have the foundation for the site right there with you, this tutorial will give you some great tips on:

  • Creating a custom theme
  • Adding customization features to your website so that changes can be made at all stages
  • Making your website more responsive and compatible

Mastering Magento

Mastering Magento is another advanced guide for those who wish to:
Get free rein over customizing their Magento website for enhanced performance
Get acquainted with the some gems of tools and techniques that are yet to catch the fancy of a large number of developers.
Grasp a clear-eyed understanding on how to make the web store run faster

Magento For Developers: Product Configuration And “Composite Products” Functionality

As the name suggests, this tutorial delves deep into the Magento territory and elucidates some complex concepts that form the very core of Magento. Authored by Andrey Tserkus, it explains the business side of things in a more detailed manner and throws light on the product configuration concepts like how to use the buyRequest, File Cope Queue, etc. You also get detailed insight on the composite products and learn how theme customizations and carts customizations can be carried out in a seamless and effective manner.

Magento 1.4 Theme Design (PDF)

This tutorial focuses inherently on the designing aspects of the Magento stores. It helps you with tips on

  • Setting your brand apart with unique and exclusive designing elements
  • Get your brand message across with elan
  • Get an absolute control over your website’s look and feel like the widgets, the header, footer, sidebar positions and the overall layout.
  • Embed social media buttons to get people talking about it on platforms like Twitter and Facebook

Magento Tutorials On Udemy

Udemy is a marketplace to find cheap tutorials. There are few Magento Tutorials available on Udemy. Some of them are really cheap priced video tutorials.

Udemy offers 30 days money back guarantee, therefore you may want to try and claim your refund if the training is not upto your expectations. I prefer to choose training with good reviews by several people, this ensures I do not waste my time and energy on a low quality provider.

Best Kindle eBooks About Magento on Amazon

The Definitive Guide to Magento

The Definitive Guide to Magento(By: Adam McCombs & Robert Banh )

When you wish to learn how to create a website from the ground up, The Definitive Guide to Magento will be a perfect ally. The guide takes a detailed look into the most basic as well as the advanced concepts of Magento and makes sure you gain an insight that goes a great deal in helping you to set your website up in quickest time possible. What sets this guide apart from the herd is that it gives you tips following which you do not have to resort to complex coding for creating your site.

Magento 1.8 Development Cookbook

Magento 1.8 Development Cookbook(By: Bart Delvaux & Nurul Ferdous )

An exceptionally useful tutorial that will empower you with the ability to tailor your Magento store for better appearance and overall functionality, this one explains the database concepts, events handlers, Cronjobs and other Magento site specifics in the most elucidatory way.

Magento 1.3 Sales Tactics Cookbook

Magento 1.3 Sales Tactics Cookbook(By: William Rice )

Written with special consideration to reader-friendly aspect,this book tosses at you a number of snapshots in order to get a visual understanding of how a particular function works and improves the site’s overall operation.

Best Video Tutorials To Learn Magento

This is a list of some of the best video tutorials that can be really handy for people who prefer video learning over reading.

  1. Magento Community Tutorials On Youtube
  2. Magento Admin Panel Tutorial
  3. Step By Step Magento Installation Tutorial (on Godaddy)
    This tutorial covers the installation of Magento on GoDaddy hosting provider.
  4. Youtube Playlist Of Various Magento Tutrials
    A tutorials compilation from different channels on a youtube playlist.
  5. Magento Tutorials Playlist By WebHunk
    This playlist contains tutorials for beginners.
  6. Video Tutorials On Magento.com
    Yet another website that publishes magento news, articles and documents for users and admins.

Useful Magento Websites and Tutorials

  1. How To Setup Magento Store This is a simple tutorial on our site for setting up multiple magento stores.
  2. Magento Tutorials By Siteground
    SiteGround is a popular magento hosting provider and also has good collection of tutorials for magento users.
  3. Official Magento Knowledge Base
    Magento Commerce is the official knowledge base for magento related tutorials, guides and documents.
  4. Magento Tutorial Series By Tuts+
    TutsPlus publishes one of really high quality tutorials and this series of tutorials are targeted for designers to get the best out of magento.
  5. Magento Tutorial
    A dedicated website for magento related news and articles.
  6. Tutorialmagento.com This site also publishes magento related tutorials and code samples.
  7. Magento Community Tuts By Levelup Tuts
    Big Collection of Magento tutorials.
  8. Getting Started On Magento
    A Beginners tutorial on TutsPlus
  9. Magento Articles On Smashing Magazine
    Smashing Magazine frequently publishes quality articles and tutorials for magento users and developers.
  10. Magento Tutorials By Yireo
    Huge collection of Magento tutorials on administration, performance and programming in magento.
  11. Magento Tutorials For Developers
    These are magento articles and tutorials targeted towards magento developers and programmers.
  12. Magento Tuts By Template Monster
    Template Monster is a popular place to buy templates, but also contains useful tutorials for magento users.
  13. Magentotutorial.net
    A dedicated blog for Magento related articles and tutorials.
  14. Magento Tips
    This site contains simple tips about magento.
  15. Magebase
    This site publishes useful articles about Magento.
  16. Magento Tutorial On FishPig
    Magento tutorials by fishpig website.
  17. Magento Tutorials By TMC Hosting
    TMD Hosting is a magento hosting solution. It also provides few tutorials.
  18. Magentotutorials.co.uk
    A few good Magento Tutorials can be found on this site as well.
  19. Magenticians
    A dedicated website for tutorials and articles on magento developers and users.
  20. LearnMagento.org
    Magento tips and tricks are published on this site.
  21. Understandinge.com
    Magento tutorials for developers and admins.
  22. Magento SEO Tutorial
    SEO Tips tutorial for optimizing your ecommerce store for best search engine results.
  23. Pixafy.com
    List of magento tutorials.
  24. AmpersandCommerce.com
    Magento tutorials by Ampersand Commerce
  25. DemacMedia.com
    Another website with magento tutorials.
  26. SolvingMagento.com
    Magento Tutorials on Solving Magento

Best Magento Forums & Discussion Boards To Ask Questions and Get Help

Learning thru tutorials and lectures may not always be sufficient to complete your daily task on magento. Sometimes you are stuck and need help. Here are some websites where you can search and ask for help.

  1. Official Magento Discussion Board This is the official magento discussion forum where most magento users, admins and developers will ask and answer questions.
  2. MagentoForum This is another popular place to get help on magento related issues.
  3. Stackoverflow Magento Tag Stackoverflow is a place for programmers to get answers. Many people ask magento related questions and you may find answers in a old thread. You can easily ask a question if you can not find a answer for your problem.
  4. Magento Experts On Freelancer Website Freelancer.com is my favorite place to find a economic help on magento related problems. There are hundreds of expert magento developers available on this site and ready work on your job for really low price. Sometimes when you are stuck and not getting much help on forums, you can try this option.

Other Useful Resources & Cheatsheets

Some other useful resources and cheatsheets that magento developers may find handy.

  1. Magento Addon Site This is official magento addon site where you can find thousands of magento addons to enhance the capability of magento store. Many addons are free.
  2. Magento: The Right Way A detailed guideline on how development needs to be done using magento.
  3. Magento Cheatsheet A beautifully laid out cheatsheet for magento developers.
  4. Magento Template Development Cheatsheet This a small cheatsheet that can be helpful for magento template developers
  5. Magento Events Cheatsheet It may be difficult to remember all the events used by the event hooking system of magento. This cheatsheet is a handy one pager for lookup.

Wrapping Up

Taking into account the core essentials of Magento and blending them with avant-garde concepts, the afore-mentioned tutorials deliver exactly what you are looking for as a learner.

(转)25 Hardening Security Tips for Linux Servers

Standard

http://www.tecmint.com/linux-server-hardening-security-tips/

 

Everybody says that Linux is secure by default and agreed to some extend (It’s debatable topics). However, Linux has in-built security model in place by default. Need to tune it up and customize as per your need which may help to make more secure system. Linux is harder to manage but offers more flexibility and configuration options.

Securing a system in a production from the hands of hackers and crackers is a challenging task for a System Administrator. This is our first article related to “How to Secure Linux box” or “Hardening a Linux Box“. In this post We’ll explain 25 useful tips & tricks to secure your Linux system. Hope, below tips & tricks will help you some extend to secure your system.

1. Physical System Security

Configure the BIOS to disable booting from CD/DVD, External Devices, Floppy Drive in BIOS. Next, enable BIOS password & also protect GRUB with password to restrict physical access of your system.

  1. Set GRUB Password to Protect Linux Servers

2. Disk Partitions

It’s important to have different partitions to obtain higher data security in case if any disaster happens. By creating different partitions, data can be separated and grouped. When an unexpected accident occurs, only data of that partition will be damaged, while the data on other partitions survived. Make sure you must have following separate partitions and sure that third party applications should be installed on separate file systems under /opt.

/
/boot
/usr
/var
/home
/tmp
/opt

3. Minimize Packages to Minimize Vulnerability

Do you really want all sort of services installed?. It’s recommended to avoid installing useless packages to avoid vulnerabilities in packages. This may minimize risk that compromise of one service may lead to compromise of other services. Find and remove or disable unwanted services from the server to minimize vulnerability. Use the ‘chkconfig‘ command to find out services which are running on runlevel 3.

# /sbin/chkconfig --list |grep '3:on'

Once you’ve find out any unwanted service are running, disable them using the following command.

# chkconfig serviceName off

Use the RPM package manager such as “yum” or “apt-get” tools to list all installed packages on a system and remove them using the following command.

# yum -y remove package-name
# sudo apt-get remove package-name
  1. 5 chkconfig Command Examples
  2. 20 Practical Examples of RPM Commands
  3. 20 Linux YUM Commands for Linux Package Management
  4. 25 APT-GET and APT-CACHE Commands to Manage Package Management

4. Check Listening Network Ports

With the help of ‘netstat‘ networking command you can view all open ports and associated programs. As I said above use ‘chkconfig‘ command to disable all unwanted network services from the system.

# netstat -tulpn
  1. 20 Netstat Commands for Network Management in Linux

5. Use Secure Shell(SSH)

Telnet and rlogin protocols uses plain text, not encrypted format which is the security breaches. SSH is a secure protocol that use encryption technology during communication with server.

Never login directly as root unless necessary. Use “sudo” to execute commands. sudo are specified in /etc/sudoers file also can be edited with the “visudo” utility which opens in VI editor.

It’s also recommended to change default SSH 22 port number with some other higher level port number. Open the main SSH configuration file and make some following parameters to restrict users to access.

# vi /etc/ssh/sshd_config
Disable root Login
PermitRootLogin no
Only allow Specific Users
AllowUsers username
Use SSH Protocol 2 Version
Protocol 2
  1. 5 Best Practices to Secure and Protect SSH Server

6. Keep System updated

Always keep system updated with latest releases patches, security fixes and kernel when it’s available.

# yum updates
# yum check-update

7. Lockdown Cronjobs

Cron has it’s own built in feature, where it allows to specify who may, and who may not want to run jobs. This is controlled by the use of files called /etc/cron.allow and /etc/cron.deny. To lock a user using cron, simply add user names in cron.deny and to allow a user to run cron add in cron.allow file. If you would like to disable all users from using cron, add the ‘ALL‘ line to cron.deny file.

# echo ALL >>/etc/cron.deny
  1. 11 Cron Scheduling Examples in Linux

8. Disable USB stick to Detect

Many times it happens that we want to restrict users from using USB stick in systems to protect and secure data from stealing. Create a file ‘/etc/modprobe.d/no-usb‘ and adding below line will not detect USB storage.

install usb-storage /bin/true

9. Turn on SELinux

Security-Enhanced Linux (SELinux) is a compulsory access control security mechanism provided in the kernel. Disabling SELinux means removing security mechanism from the system. Think twice carefully before removing, if your system is attached to internet and accessed by the public, then think some more on it.

SELinux provides three basic modes of operation and they are.

  1. Enforcing: This is default mode which enable and enforce the SELinux security policy on the machine.
  2. Permissive: In this mode, SELinux will not enforce the security policy on the system, only warn and log actions. This mode is very useful in term of troubleshooting SELinux related issues.
  3. Disabled: SELinux is turned off.

You can view current status of SELinux mode from the command line using ‘system-config-selinux‘, ‘getenforce‘ or ‘sestatus‘ commands.

# sestatus

If it is disabled, enable SELinux using the following command.

# setenforce enforcing

It also can be managed from ‘/etc/selinux/config‘ file, where you can enable or disable it.

10. Remove KDE/GNOME Desktops

There is no need to run X Window desktops like KDE or GNOME on your dedicated LAMP server. You can remove or disable them to increase security of server and performance. To disable simple open the file ‘/etc/inittab‘ and set run level to 3. If you wish to remove it completely from the system use the below command.

# yum groupremove "X Window System"

11. Turn Off IPv6

If you’re not using a IPv6 protocol, then you should disable it because most of the applications or policies not required IPv6 protocol and currently it doesn’t required on the server. Go to network configuration file and add followings lines to disable it.

# vi /etc/sysconfig/network
NETWORKING_IPV6=no
IPV6INIT=no

12. Restrict Users to Use Old Passwords

This is very useful if you want to disallow users to use same old passwords. The old password file is located at /etc/security/opasswd. This can be achieved by using PAM module.

Open ‘/etc/pam.d/system-auth‘ file under RHEL / CentOS / Fedora.

# vi /etc/pam.d/system-auth

Open ‘/etc/pam.d/common-password‘ file under Ubuntu/Debian/Linux Mint.

# vi /etc/pam.d/common-password

Add the following line to ‘auth‘ section.

auth        sufficient    pam_unix.so likeauth nullok

Add the following line to ‘password‘ section to disallow a user from re-using last 5 password of his or her.

password   sufficient    pam_unix.so nullok use_authtok md5 shadow remember=5

Only last 5 passwords are remember by server. If you tried to use any of last 5 old passwords, you will get an error like.

Password has been already used. Choose another.

13. How to Check Password Expiration of User

In Linux, user’s passwords are stored in ‘/etc/shadow‘ file in encrypted format. To check password expiration of user’s, you need to use ‘chage‘ command. It displays information of password expiration details along with last password change date. These details are used by system to decide when a user must change his/her password.

To view any existing user’s aging information such as expiry date and time, use the following command.

#chage -l username

To change password aging of any user, use the following command.

#chage -M 60 username
#chage -M 60 -m 7 -W 7 userName
Parameters
  1. -M Set maximum number of days
  2. -m Set minimum number of days
  3. -W Set the number of days of warning

14. Lock and Unlock Account Manually

The lock and unlock features are very useful, instead of removing an account from the system, you can lock it for an week or a month. To lock a specific user, you can use the follow command.

# passwd -l accountName

Note : The locked user is still available for root user only. The locking is performed by replacing encrypted password with an (!) string. If someone trying to access the system using this account, he will get an error similar to below.

# su - accountName
This account is currently not available.

To unlock or enable access to an locked account, use the command as. This will remove (!) string with encrypted password.

# passwd -u accountName

15. Enforcing Stronger Passwords

A number of users use soft or weak passwords and their password might be hacked with a dictionary based or brute-force attacks. The ‘pam_cracklib‘ module is available in PAM (Pluggable Authentication Modules) module stack which will force user to set strong passwords. Open the following file with an editor.

Read Also:

# vi /etc/pam.d/system-auth

And add line using credit parameters as (lcredit, ucredit, dcredit and/or ocredit respectively lower-case, upper-case, digit and other)

/lib/security/$ISA/pam_cracklib.so retry=3 minlen=8 lcredit=-1 ucredit=-2 dcredit=-2 ocredit=-1

16. Enable Iptables (Firewall)

It’s highly recommended to enable Linux firewall to secure unauthorised access of your servers. Apply rules in iptables to filters incoming, outgoing and forwarding packets. We can specify the source and destination address to allow and deny in specific udp/tcp port number.

  1. Basic IPTables Guide and Tips

17. Disable Ctrl+Alt+Delete in Inittab

In most Linux distributions, pressing ‘CTRL-ALT-DELETE’ will takes your system to reboot process. So, it’s not a good idea to have this option enabled at least on production servers, if someone by mistakenly does this.

This is defined in ‘/etc/inittab‘ file, if you look closely in that file you will see a line similar to below. By default line is not commented out. We have to comment it out. This particular key sequence signalling will shut-down a system.

# Trap CTRL-ALT-DELETE
#ca::ctrlaltdel:/sbin/shutdown -t3 -r now

18. Checking Accounts for Empty Passwords

Any account having an empty password means its opened for unauthorized access to anyone on the web and it’s a part of security within a Linux server. So, you must make sure all accounts have strong passwords and no one has any authorized access. Empty password accounts are security risks and that can be easily hackable. To check if there were any accounts with empty password, use the following command.

# cat /etc/shadow | awk -F: '($2==""){print $1}'

19. Display SSH Banner Before Login

It’s always a better idea to have an legal banner or security banners with some security warnings before SSH authentication. To set such banners read the following article.

  1. Display SSH Warning Message to Users

20. Monitor User Activities

If you are dealing with lots of users, then its important to collect the information of each user activities and processes consumed by them and analyse them at a later time or in case if any kind of performance, security issues. But how we can monitor and collect user activities information.

There are two useful tools called ‘psacct‘ and ‘acct‘ are used for monitoring user activities and processes on a system. These tools runs in a system background and continuously tracks each user activity on a system and resources consumed by services such as Apache, MySQL, SSH, FTP, etc. For more information about installation, configuration and usage, visit the below url.

  1. Monitor User Activity with psacct or acct Commands

21. Review Logs Regularly

Move logs in dedicated log server, this may prevents intruders to easily modify local logs. Below are the Common Linux default log files name and their usage:

  1. /var/log/message – Where whole system logs or current activity logs are available.
  2. /var/log/auth.log – Authentication logs.
  3. /var/log/kern.log – Kernel logs.
  4. /var/log/cron.log – Crond logs (cron job).
  5. /var/log/maillog – Mail server logs.
  6. /var/log/boot.log – System boot log.
  7. /var/log/mysqld.log – MySQL database server log file.
  8. /var/log/secure – Authentication log.
  9. /var/log/utmp or /var/log/wtmp : Login records file.
  10. /var/log/yum.log: Yum log files.

22. Important file Backup

In a production system, it is necessary to take important files backup and keep them in safety vault, remote site or offsite for Disasters recovery.

23. NIC Bonding

There are two types of mode in NIC bonding, need to mention in bonding interface.

  1. mode=0 – Round Robin
  2. mode=1 – Active and Backup

NIC Bonding helps us to avoid single point of failure. In NIC bonding, we bond two or more Network Ethernet Cards together and make one single virtual Interface where we can assign IP address to talk with other servers. Our network will be available in case of one NIC Card is down or unavailable due to any reason.

Read Also : Create NIC Channel Bonding in Linux

24. Keep /boot as read-only

Linux kernel and its related files are in /boot directory which is by default as read-write. Changing it to read-only reduces the risk of unauthorized modification of critical boot files. To do this, open “/etc/fstab” file.

# vi /etc/fstab

Add the following line at the bottom, save and close it.

LABEL=/boot     /boot     ext2     defaults,ro     1 2

Please note that you need to reset the change to read-write if you need to upgrade the kernel in future.

25. Ignore ICMP or Broadcast Request

Add following line in “/etc/sysctl.conf” file to ignore ping or broadcast request.

Ignore ICMP request:
net.ipv4.icmp_echo_ignore_all = 1

Ignore Broadcast request:
net.ipv4.icmp_echo_ignore_broadcasts = 1

Load new settings or changes, by running following command

#sysctl -p

If you’ve missed any important security or hardening tip in the above list, or you’ve any other tip that needs to be included in the list. Please drop your comments in our comment box. TecMint is always interested in receiving comments, suggestions as well as discussion for improvement.

 

(FW)深入解析NoSQL数据库的分布式算法

Standard

http://www.csdn.net/article/2015-01-30/2823782

原文是Ilya Katsov大神写,给个链接https://highlyscalable.wordpress.com/2012/09/18/distributed-algorithms-in-nosql-databases/

 

系统的可扩展性是推动NoSQL运动发展的的主要理由,包含了分布式系统协调,故障转移,资源管理和许多其他特性。这么讲使得NoSQL听起来像是 一个大筐,什么都能塞进去。尽管NoSQL运动并没有给分布式数据处理带来根本性的技术变革,但是依然引发了铺天盖地的关于各种协议和算法的研究以及实 践。正是通过这些尝试逐渐总结出了一些行之有效的数据库构建方法。在这篇文章里,我将针对NoSQL数据库的分布式特点进行一些系统化的描述。

接下来我们将研究一些分布式策略,比如故障检测中的复制,这些策略用黑体字标出,被分为三段:

  • 数据一致性。NoSQL需要在分布式系统的一致性,容错性和性能,低延迟及高可用之间作出权衡,一般来说,数据一致性是一个必选项,所以这一节主要是关于数据复制数据恢复
  • 数据放置。一个数据库产品应该能够应对不同的数据分布,集群拓扑和硬件配置。在这一节我们将讨论如何分布以及调整数据分布才能够能够及时解决故障,提供持久化保证,高效查询和保证集群中的资源(如内存和硬盘空间)得到均衡使用。
  • 对等系统。像 leader election 这样的的技术已经被用于多个数据库产品以实现容错和数据强一致性。然而,即使是分散的的数据库(无中心)也要跟踪它们的全局状态,检测故障和拓扑变化。这一节将介绍几种使系统保持一致状态的技术。

数据一致性

众 所周知,分布式系统经常会遇到网络隔离或是延迟的情况,在这种情况下隔离的部分是不可用的,因此要保持高可用性而不牺牲一致性是不可能的。这一事实通常被 称作“CAP理论”。然而,一致性在分布式系统中是一个非常昂贵的东西,所以经常需要在这上面做一些让步,不只是针对可用性,还有多种权衡。为了研究这些 权衡,我们注意到分布式系统的一致性问题是由数据隔离和复制引起的,所以我们将从研究复制的特点开始:

  • 可用性。在网络隔离的情况下剩余部分仍然可以应对读写请求。
  • 读写延迟。读写请求能够在短时间内处理。
  • 读写延展性。读写的压力可由多个节点均衡分担。
  • 容错性。对于读写请求的处理不依赖于任何一个特定节点。
  • 数据持久性。特定条件下的节点故障不会造成数据丢失。
  • 一致性。一致性比前面几个特性都要复杂得多,我们需要详细讨论一下几种不同的观点。 但是我们不会涉及过多的一致性理论和并发模型,因为这已经超出了本文的范畴,我只会使用一些简单特点构成的精简体系。
  • 读写一致性。从读写的观点来看,数据库的基本目标是使副本趋同的时间尽可能短(即更新传递到所有副本的时间),保证最终一致性。除了这个较弱的保证,还有一些更强的一致性特点:
  • 写后读一致性。在数据项X上写操作的效果总是能够被后续的X上的读操作看见。
  • 读后读一致性。在一次对数据项X的读操作之后,后续对X的读操作应该返回与第一次的返回值相同或是更加新的值。
  • 写一致性。分区的数据库经常会发生写冲突。数据库应当能处理这种冲突并保证多个写请求不会被不同的分区所处理。这方面数据库提供了几种不同的一致性模型:
  • 原子写。假 如数据库提供了API,一次写操作只能是一个单独的原子性的赋值,避免写冲突的办法是找出每个数据的“最新版本”。这使得所有的节点都能够在更新结束时获 得同一版本,而与更新的顺序无关,网络故障和延迟经常造成各节点更新顺序不一致。 数据版本可以用时间戳或是用户指定的值来表示。Cassandra用的就是这种方法。
  • 原子化的读-改-写。应 用有时候需要进行 读-改-写 序列操作而非单独的原子写操作。假如有两个客户端读取了同一版本的数据,修改并且把修改后的数据写回,按照原子写模型,时间上比较靠后的那一次更新将会覆 盖前一次。这种行为在某些情况下是不正确的(例如,两个客户端往同一个列表值中添加新值)。数据库提供了至少两种解决方法:
  • 冲突预防。 读-改-写 可以被认为是一种特殊情况下的事务,所以分布式锁或是 PAXOS这样的一致协议都可以解决这种问题。这种技术支持原子读改写语义和任意隔离级别的事务。另一种方法是避免分布式的并发写操作,将对特定数据项的 所有写操作路由到单个节点上(可以是全局主节点或者分区主节点)。为了避免冲突,数据库必须牺牲网络隔离情况下的可用性。这种方法常用于许多提供强一致性 保证的系统(例如大多数关系数据库,HBase,MongoDB)。
  • 冲突检测。数据库跟踪并发更新的冲突,并选择回滚其中之一或是维持两个版本交由客户端解决。并发更新通常用向量时钟 (这是一种乐观锁)来跟踪,或者维护一个完整的版本历史。这个方法用于 Riak, Voldemort, CouchDB.

现在让我们仔细看看常用的复制技术,并按照描述的特点给他们分一下类。第一幅图描绘了不同技术之间的逻辑关系和不同技术在系统的一致性、扩展性、可用性、延迟性之间的权衡坐标。 第二张图详细描绘了每个技术。

复本因子是4。读写协调者可以是一个外部客户端或是一个内部代理节点。

我们会依据一致性从弱到强把所有的技术过一遍:

(A, 反熵) 一致性最弱,基于策略如下。写操作的时候选择任意一个节点更新,在读的时候如果新数据还没有通过后台的反熵协议传递到读的那个节点,那么读到的仍然是旧数据。(下一节会详细介绍反熵协议)。这种方法的主要特点是:

  • 过高的传播延迟使它在数据同步方面不太好用,所以比较典型的用法是只作为辅助性的功能来检测和修复计划外的不一致。Cassandra就使用了反熵算法来在各节点之间传递数据库拓扑和其他一些元数据信息。
  • 一致性保证较弱:即使在没有发生故障的情况下,也会出现写冲突与读写不一致。
  • 在网络隔离下的高可用和健壮性。用异步的批处理替代了逐个更新,这使得性能表现优异。
  • 持久性保障较弱因为新的数据最初只有单个副本。

(B) 对上面模式的一个改进是在任意一个节点收到更新数据请求的同时异步的发送更新给所有可用节点。这也被认为是定向的反熵。

  • 与纯粹的反熵相比,这种做法只用一点小小的性能牺牲就极大地提高了一致性。然而,正式一致性和持久性保持不变。
  • 假如某些节点因为网络故障或是节点失效在当时是不可用的,更新最终也会通过反熵传播过程来传递到该节点。

(C) 在前一个模式中,使用提示移交技术可以更好地处理某个节点的操作失败。对于失效节点的预期更新被记录在额外的代理节点上,并且标明一旦特点节点可用就要将更新传递给该节点。这样做提高了一致性,降低了复制收敛时间。

(D, 一次性读写)因为提示移交的责任节点也有可能在将更新传递出去之前就已经失效,在这种情况下就有必要通过所谓的读修复来保证一致性。每 个读操作都会启动一个异步过程,向存储这条数据的所有节点请求一份数据摘要(像签名或者hash),如果发现各节点返回的摘要不一致则统一各节点上的数据 版本。我们用一次性读写来命名组合了A、B、C、D的技术- 他们都没有提供严格的一致性保证,但是作为一个自备的方法已经可以用于实践了。

(E, 读若干写若干) 上面的策略是降低了复制收敛时间的启发式增强。为了保证更强的一致性,必须牺牲可用性来保证一定的读写重叠。 通常的做法是同时写入W个副本而不是一个,读的时候也要读R个副本。

  • 首先,可以配置写副本数W>1。
  • 其 次,因为R+W>N,写入的节点和读取的节点之间必然会有重叠,所以读取的多个数据副本里至少会有一个是比较新的数据(上面的图中 W=2, R=3, N=4 )。这样在读写请求依序进行的时候(写执行完再读)能够保证一致性(对于单个用户的读写一致性),但是不能保障全局的读一致性。用下面图示里的例子来 看,R=2,W=2,N=3,因为写操作对于两个副本的更新是非事务的,在更新没有完成的时候读就可能读到两个都是旧值或者一新一旧:

  • 对于某种读延迟的要求,设置R和W的不同值可以调整写延迟与持久性,反之亦然。
  • 如果W<=N/2,并发的多个写入会写到不同的若干节点(如,写操作A写前N/2个,B写后N/2个)。 设置 W>N/2 可以保证在符合回滚模型的原子读改写时及时检测到冲突。
  • 严格来讲,这种模式虽然可以容忍个别节点的失效, 但是对于网络隔离的容错性并不好。在实践中,常使用”近似数量通过“这样的方法,通过牺牲一致性来提高某些情景下的可用性。

(F, 读全部写若干)读一致性问题可以通过在读数据的时候访问所有副本(读数据或者检查摘要)来减轻。这确保了只要有至少一个节点上的数据更新新的数据就能被读取者看到。但是在网络隔离的情况下这种保证就不能起到作用了。

(G, 主从) 这种技术常被用来提供原子写或者 冲突检测持久级别的读改写。为了实现冲突预防级别,必须要用一种集中管理方式或者是锁。最简单的策略是用主从异步复制。对于特定数据项的写操作全部被路由到一个中心节点,并在上面顺序执行。这种情况下主节点会成为瓶颈,所以必须要将数据划分成一个个独立的片区(不同片有不同的master),这样才能提供扩展性。

(H, Transactional Read Quorum Write Quorum and Read One Write All)  更新多个副本的方法可以通过使用事务控制技术来避免写冲突。 众所周知的方法是使用两阶段提交协议。但两阶段提交并不是完全可靠的,因为协调者失效可能会造成资源阻塞。 PAXOS提交协议是更可靠的选择,但会损失一点性能。 在这个基础上再向前一小步就是读一个副本写所有副本,这种方法把所有副本的更新放在一个事务中,它提供了强容错一致性但会损失掉一些性能和可用性。

上面分析中的一些权衡有必要再强调一下

  • 一致性与可用性。 严密的权衡已经由CAP理论给出了。在网络隔离的情况下,数据库要么将数据集中,要么既要接受数据丢失的风险。
  • 一 致性与扩展性。 看得出即使读写一致性保证降低了副本集的扩展性,只有在原子写模型中才可以以一种相对可扩展的方式处理写冲突。原子读改写模型通过给数据加上临时性的全局 锁来避免冲突。这表明, 数据或操作之间的依赖,即使是很小范围内或很短时间的,也会损害扩展性。所以精心设计数据模型,将数据分片分开存放对于扩展性非常重要。
  • 一致性与延迟。 如上所述,当数据库需要提供强一致性或者持久性的时候应该偏向于读写所有副本技术。但是很明显一致性与请求延迟成反比,所以使用若干副本技术会是比较中允的办法。
  • 故 障转移与一致性/扩展性/延迟。有趣的是容错性与一致性、扩展性、延迟的取舍冲突并不剧烈。通过合理的放弃一些性能与一致性,集群可以容忍多达 up to 的节点失效。这种折中在两阶段提交与 PAXOS 协议的区别里体现得很明显。这种折中的另一个例子是增加特定的一致性保障,比如使用严格会话进程的“读己所写”,但这又增加了故障转移的复杂性。

反熵协议, 谣言传播算法

让我们从以下场景开始:

有 许多节点,每条数据会在其中的若干的节点上面存有副本。每个节点都可以单独处理更新请求,每个节点定期和其他节点同步状态,如此一段时间之后所有的副本都 会趋向一致。同步过程是怎样进行的?同步何时开始?怎样选择同步的对象?怎么交换数据?我们假定两个节点总是用较新版本的数据覆盖旧的数据或者两个版本都 保留以待应用层处理。

这个问题常见于数据一致性维护和集群状态同步(如集群成员信息传播)等场景。虽然引入一个监控数据库并制定同步计划的 协调者可以解决这个问题,但是去中心化的数据库能够提供更好的容错性。去中心化的主要做法是利用精心设计的传染协议,这种协议相对简单,但是提供了很好的 收敛时间,而且能够容忍任何节点的失效和网络隔离。尽管有许多类型的传染算法,我们只关注反熵协议,因为NoSQL数据库都在使用它。

反熵 协议假定同步会按照一个固定进度表执行,每个节点定期随机或是按照某种规则选择另外一个节点交换数据,消除差异。有三种反风格的反熵协议:推,拉和混合。 推协议的原理是简单选取一个随机节点然后把数据状态发送过去。在真实应用中将全部数据都推送出去显然是愚蠢的,所以节点一般按照下图所示的方式工作。

节点A作为同步发起者准备好一份数据摘要,里面包含了A上数据的指纹。节点B接收到摘要之后将摘要中的数据与本地数据进行比较,并将数据差异做成一份摘要返回给A。最后,A发送一个更新给B,B再更新数据。拉方式和混合方式的协议与此类似,就如上图所示的。

反熵协议提供了足够好的收敛时间和扩展性。下图展示了一个在100个节点的集群中传播一个更新的模拟结果。在每次迭代中,每个节点只与一个随机选取的对等节点发生联系。

可 以看到,拉方式的收敛性比推方式更好,这可以从理论上得到证明。而且推方式还存在一个“收敛尾巴”的问题。在多次迭代之后,尽管几乎遍历到了所有的节点, 但还是有很少的一部分没受到影响。与单纯的推和拉方式相比, 混合方式的效率更高,所以实际应用中通常使用这种方式。反熵是可扩展的,因为平均转换时间以集群规模的对数函数形式增长。

尽管这些技术看起 来很简单,仍然有许多研究关注于不同约束条件下反熵协议的性能表现。其中之一通过一种更有效的结构使用网络拓扑来取代随机选取 。在网络带宽有限的条件下调整传输率或使用先进的规则来选取要同步的数据 。摘要计算也面临挑战,数据库会维护一份最近更新的日志以有助于摘要计算。

最终一致数据类型Eventually Consistent Data Types

在上一节我们假定两个节点总是合并他们的数据版本。但要解决更新冲突并不容易,让所有副本都最终达到一个语义上正确的值出乎意料的难。一个众所周知的例子是Amazon Dynamo数据库中已经删除的条目可以重现。

我 们假设一个例子来说明这个问题:数据库维护一个逻辑上的全局计数器,每个节点可以增加或者减少计数。虽然每个节点可以在本地维护一个自己的值,但这些本地 计数却不能通过简单的加减来合并。假设这样一个例子:有三个节点A、B和C,每个节点执行了一次加操作。如果A从B获得一个值,并且加到本地副本上,然后 C从B获得值,然后C再从A获得值,那么C最后的值是4,而这是错误的。解决这个问题的方法是用一个类似于向量时钟的数据结构为每个节点维护一对计数器:

  1. class Counter {
  2. 2     int[] plus
  3. 3     int[] minus
  4. 4     int NODE_ID
  5. 5
  6. 6     increment() {
  7. 7         plus[NODE_ID]++
  8. 8     }
  9. 9
  10. 10    decrement() {
  11. 11        minus[NODE_ID]++
  12. 12    }
  13. 13
  14. 14    get() {
  15. 15        return sum(plus) – sum(minus)
  16. 16    }
  17. 17
  18. 18    merge(Counter other) {
  19. 19        for i in 1..MAX_ID {
  20. 20            plus[i] = max(plus[i], other.plus[i])
  21. 21            minus[i] = max(minus[i], other.minus[i])
  22. 22        }
  23. 23    }
  24. 24 }

Cassandra用类似的方法计数。利用基于状态的或是基于操作的复制理论也可以设计出更复杂的最终一致的数据结构。例如,中就提及了一系列这样的数据结构,包括:

  • 计数器(加减操作)
  • 集合(添加和移除操作)
  • 图(增加边或顶点,移除边或顶点)
  • 列表(插入某位置或者移除某位置)

最终一致数据类型的功能通常是有限的,还会带来额外的性能开销。

数据放置

这部分主要关注控制在分布式数据库中放置数据的算法。这些算法负责把数据项映射到合适的物理节点上,在节点间迁移数据以及像内存这样的资源的全局调配。

均衡数据

我 们还是从一个简单的协议开始,它可以提供集群节点间无缝的数据迁移。这常发生于像集群扩容(加入新节点),故障转移(一些节点宕机)或是均衡数据(数据在 节点间的分布不均衡)这样的场景。如下图A中所描绘的场景 – 有三个节点,数据随便分布在三个节点上(假设数据都是key-value型)。

如 果数据库不支持数据内部均衡,就要在每个节点上发布数据库实例,如上面图B所示。这需要手动进行集群扩展,停掉要迁移的数据库实例,把它转移到新节点上, 再在新节点上启动,如图C所示。尽管数据库能够监控到每一条记录,包括MongoDB, Oracle Coherence, 和还在开发中的 Redis Cluster 在内的许多系统仍然使用的是自动均衡技术。也即,将数据分片并把每个数据分片作为迁移的最小单位,这是基于效率的考虑。很明显分片数会比节点数多,数据分 片可以在各节点间平均分布。按照一种简单的协议即可实现无缝数据迁移,这个协议可以在迁移数据分片的时候重定向客户的数据迁出节点和迁入节点。下图描绘了 一个Redis Cluster中实现的get(key)逻辑的状态机。

假 定每个节点都知道集群拓扑,能够把任意key映射到相应的数据分片,把数据分片映射到节点。如果节点判断被请求的key属于本地分片,就会在本地查找(上 图中上面的方框)。假如节点判断请求的key属于另一个节点X,他会发送一个永久重定向命令给客户端(上图中下方的方框)。永久重定向意味着客户端可以缓 存分片和节点间的映射关系。如果分片迁移正在进行,迁出节点和迁入节点会标记相应的分片并且将分片的数据加锁逐条加锁然后开始移动。迁出节点首先会在本地 查找key,如果没有找到,重定向客户端到迁入节点,假如key已经迁移完毕的话。这种重定向是一次性的,并且不能被缓存。迁入节点在本地处理重定向,但 定期查询在迁移还没完成前被永久重定向。

动态环境中的数据分片和复制

我们关注的另一个问题是怎么把记录映射到物 理节点。比较直接的方法是用一张表来记录每个范围的key与节点的映射关系,一个范围的key对应到一个节点,或者用key的hash值与节点数取模得到 的值作为节点ID。但是hash取模的方法在集群发生更改的情况下就不是很好用,因为增加或者减少节点都会引起集群内的数据彻底重排。导致很难进行复制和 故障恢复。

有许多方法在复制和故障恢复的角度进行了增强。最著名的就是一致性hash。网上已经有很多关于一致性hash的介绍了,所以在这里我只提供一个基本介绍,仅仅为了文章内容的完整性。下图描绘了一致性hash的基本原理:

一 致性hash从根本上来讲是一个键值映射结构 – 它把键(通常是hash过的)映射到物理节点。键经过hash之后的取值空间是一个有序的定长二进制字符串,很显然每个在此范围内的键都会被映射到图A中 A、B、C三个节点中的某一个。为了副本复制,将取值空间闭合成一个环,沿环顺时针前行直到所有副本都被映射到合适的节点上,如图B所示。换句话说,Y将 被定位在节点B上,因为它在B的范围内,第一个副本应该放置在C,第二个副本放置在A,以此类推。

这种结构的好处体现在增加或减少一个节点 的时候,因为它只会引起临接区域的数据重新均衡。如图C所示,节点D的加入只会对数据项X产生影响而对Y无影响。同样,移除节点B(或者B失效)只会影响 Y和X的副本,而不会对X自身造成影响。但是,这种做法在带来好处的同时也有弱点,那就是重新均衡的负担都由邻节点承受了,它们将移动大量的数据。通过将 每个节点映射到多个范围而不是一个范围可以一定程度上减轻这个问题带来的不利影响,如图D所示。这是一个折中,它避免了重新均衡数据时负载过于集中,但是 与基于模块的映射相比,保持了总均衡数量适当降低。

给大规模的集群维护一个完整连贯的hash环很不容易。对于相对小一点的数据库集群就不 会有问题,研究如何在对等网络中将数据放置与网络路由结合起来很有意思。一个比较好的例子是Chord算法,它使环的完整性让步于单个节点的查找效率。 Chord算法也使用了环映射键到节点的理念,在这方面和一致性hash很相似。不同的是,一个特定节点维护一个短列表,列表中的节点在环上的逻辑位置是 指数增长的(如下图)。这使得可以使用二分搜索只需要几次网络跳跃就可以定位一个键。

这张图画的是一个由16个节点组成的集群,描绘了节点A是如何查找放在节点D上的key的。 (A) 描绘了路由,(B) 描绘了环针对节点A、B、C的局部图像。在参考资料中有更多关于分散式系统中的数据复制的内容。

按照多个属性的数据分片

当 只需要通过主键来访问数据的时候,一致性hash的数据放置策略很有效,但是当需要按照多个属性来查询的时候事情就会复杂得多。一种简单的做法 (MongoDB使用的)是用主键来分布数据而不考虑其他属性。这样做的结果是依据主键的查询可以被路由到接个合适的节点上,但是对其他查询的处理就要遍 历集群的所有节点。查询效率的不均衡造成下面的问题:

有一个数据集,其中的每条数据都有若干属性和相应的值。是否有一种数据分布策略能够使得限定了任意多个属性的查询会被交予尽量少的几个节点执行?

HyperDex数据库提供了一种解决方案。基本思想是把每个属性视作多维空间中的一个轴,将空间中的区域映射到物理节点上。一次查询会被对应到一个由空间中多个相邻区域组成的超平面,所以只有这些区域与该查询有关。让我们看看参考资料中的一个例子:

每 一条数据都是一条用户信息,有三个属性First Name 、Last Name 和Phone Number。这些属性被视作一个三维空间,可行的数据分布策略是将每个象限映射到一个物理节点。像“First Name = John”这样的查询对应到一个贯穿4个象限的平面,也即只有4个节点会参与处理此次查询。有两个属性限制的查询对应于一条贯穿两个象限的直线,如上图所 示,因此只有2个节点会参与处理。

这个方法的问题是空间象限会呈属性数的指数函数增长。结果就会是,只有几个属性限制的查询会投射到许多个 空间区域,也即许多台服务器。将一个属性较多的数据项拆分成几个属性相对较少的子项,并将每个子项都映射到一个独立的子空间,而不是将整条数据映射到一个 多维空间,这样可以一定程度上缓解这个问题:

这样能够提供更好的查询到节点的映射,但是增加了集群协调的复杂度,因为这种情况下一条数据会散布在多个独立的子空间,而每个子空间都对应各自的若干个物理节点,数据更新时就必须考虑事务问题。

钝化副本

有 的应用有很强的随机读取要求,这就需要把所有数据放在内存里。在这种情况下,将数据分片并把每个分片主从复制通常需要两倍以上的内存,因为每个数据都要在 主节点和从节点上各有一份。为了在主节点失效的时候起到代替作用,从节点上的内存大小应该和主节点一样。如果系统能够容忍节点失效的时候出现短暂中断或性 能下降,也可以不要分片。

下面的图描绘了4个节点上的16个分片,每个分片都有一份在内存里,副本存在硬盘上:

灰 色箭头突出了节点2上的分片复制。其他节点上的分片也是同样复制的。红色箭头描绘了在节点2失效的情况下副本怎样加载进内存。集群内副本的均匀分布使得只 需要预留很少的内存就可以存放节点失效情况下激活的副本。在上面的图里,集群只预留了1/3的内存就可以承受单个节点的失效。特别要指出的是副本的激活 (从硬盘加载入内存)会花费一些时间,这会造成短时间的性能下降或者正在恢复中的那部分数据服务中断。

系统协调

在这部分我们将讨论与系统协调相关的两种技术。分布式协调是一个比较大的领域,数十年以来有很多人对此进行了深入的研究。这篇文章里只涉及两种已经投入实用的技术。关于分布式锁,consensus协议以及其他一些基础技术的内容可以在很多书或者网络资源中找到。

故障检测

故 障检测是任何一个拥有容错性的分布式系统的基本功能。实际上所有的故障检测协议都基于心跳通讯机制,原理很简单,被监控的组件定期发送心跳信息给监控进程 (或者由监控进程轮询被监控组件),如果有一段时间没有收到心跳信息就被认为失效了。除此之外,真正的分布式系统还要有另外一些功能要求:

  • 自适应。故 障检测应该能够应对暂时的网络故障和延迟,以及集群拓扑、负载和带宽的变化。但这有很大难度,因为没有办法去分辨一个长时间没有响应的进程到底是不是真的 失效了,因此,故障检测需要权衡故障识别时间(花多长时间才能识别一个真正的故障,也即一个进程失去响应多久之后会被认为是失效)和虚假警报率之间的轻 重。这个权衡因子应该能够动态自动调整。
  • 灵活性。乍 看上去,故障检测只需要输出一个表明被监控进程是否处于工作状态的布尔值,但在实际应用中这是不够的。我们来看参考资料中的一个类似MapReduce的 例子。有一个由一个主节点和若干工作节点组成的分布式应用,主节点维护一个作业列表,并将列表中的作业分配给工作节点。主节点能够区分不同程度的失败。如 果主节点怀疑某个工作节点挂了,他就不会再给这个节点分配作业。其次,随着时间推移,如果没有收到该节点的心跳信息,主节点就会把运行在这个节点上的作业 重新分配给别的节点。最后,主节点确认这个节点已经失效,并释放所有相关资源。
  • 可扩展性和健壮性。失 败检测作为一个系统功能应该能够随着系统的扩大而扩展。他应该是健壮和一致的,也即,即使在发生通讯故障的情况下,系统中的所有节点都应该有一个一致的看 法(即所有节点都应该知道哪些节点是不可用的,那些节点是可用的,各节点对此的认知不能发生冲突,不能出现一部分节点知道某节点A不可用,而另一部分节点 不知道的情况)

所谓的累计失效检测器可以解决前两个问题,Cassandra对它进行了一些修改并应用在产品中。其基本工作流程如下:

  • 对于每一个被监控资源,检测器记录心跳信息到达时间Ti。
  • 计算在统计预测范围内的到达时间的均值和方差。
  • 假 定到达时间的分布已知(下图包括一个正态分布的公式),我们可以计算心跳延迟(当前时间t_now和上一次到达时间Tc之间的差值) 的概率,用这个概率来判断是否发生故障。可以使用对数函数来调整它以提高可用性。在这种情况下,输出1意味着判断错误(认为节点失效)的概率是10%,2 意味着1%,以此类推。

根据重要程度不同来分层次组织监控区,各区域之间通过谣言传播协议或者中央容错库同步,这样可以满足扩展性的要求,又可以防止心跳信息在网络中泛滥。如下图所示(6个故障检测器组成了两个区域,互相之间通过谣言传播协议或者像ZooKeeper这样的健壮性库来联系):

协调者竞选

协调者竞选是用于强一致性数据库的一个重要技术。首先,它可以组织主从结构的系统中主节点的故障恢复。其次,在网络隔离的情况下,它可以断开处于少数的那部分节点,以避免写冲突。

Bully 算法是一种相对简单的协调者竞选算法。MongoDB 用了这个算法来决定副本集中主要的那一个。Bully 算法的主要思想是集群的每个成员都可以声明它是协调者并通知其他节点。别的节点可以选择接受这个声称或是拒绝并进入协调者竞争。被其他所有节点接受的节点 才能成为协调者。节点按照一些属性来判断谁应该胜出。这个属性可以是一个静态ID,也可以是更新的度量像最近一次事务ID(最新的节点会胜出)。

下图的例子展示了bully算法的执行过程。使用静态ID作为度量,ID值更大的节点会胜出:

  1. 最初集群有5个节点,节点5是一个公认的协调者。
  2. 假设节点5挂了,并且节点2和节点3同时发现了这一情况。两个节点开始竞选并发送竞选消息给ID更大的节点。
  3. 节点4淘汰了节点2和3,节点3淘汰了节点2。
  4. 这时候节点1察觉了节点5失效并向所有ID更大的节点发送了竞选信息。
  5. 节点2、3和4都淘汰了节点1。
  6. 节点4发送竞选信息给节点5。
  7. 节点5没有响应,所以节点4宣布自己当选并向其他节点通告了这一消息。

协 调者竞选过程会统计参与的节点数目并确保集群中至少一半的节点参与了竞选。这确保了在网络隔离的情况下只有一部分节点能选出协调者(假设网络中网络会被分 割成多块区域,之间互不联通,协调者竞选的结果必然会在节点数相对比较多的那个区域中选出协调者,当然前提是那个区域中的可用节点多于集群原有节点数的半 数。如果集群被隔离成几个区块,而没有一个区块的节点数多于原有节点总数的一半,那就无法选举出协调者,当然这样的情况下也别指望集群能够继续提供服务 了)。

原文地址:http://juliashine.com/distributed-algorithms-in-nosql-databases/

2015年你需要学习的15种编程语言

Standard

1. Java

Java用于构建企业web应用的后端,是当今最为流行的编程语言之一。Web开发人员可以凭借Java和基于Java的框架为各种用户建立可扩展的Web应用程序。 Java也是为智能手机和平板电脑开发原生Android应用程序的主要语言。

2. JavaScript

现在的每一个网站都使用了JavaScript。如果你想在你的网站上创建互动环节,或者用一些流行的JavaScript框架建立用户界面,那么JavaScript就是首选。

3.C#

C#是开发微软平台和服务的主要语言。无论你是使用Azure和.NET构建现代化的web应用,还是为Windows设备构建app,亦或是为企 业创建功能强大的桌面应用,C#都是驾驭微软的最快方式。想开发游戏?流行的统一游戏开发引擎也使用了C#作为其主要的编程语言之一。

4. PHP

需要构建操作数据的Web应用? 那么,PHP以及MySQL之类的数据库,是我们必不可少的工具。 PHP活跃于大多数数据驱动的网站,并且是作为内容管理系统的基础性技术,内容管理系统包括Wordpress,它能扩展你的网站功能,使之更为强大。

5. C ++

如果你需要直接连接到硬件以获得最大的处理能力,那么C++就是最佳选择。它可以用于开发功能强大的桌面软件、硬件加速的游戏和内存密集型的app。

6. Python

Web应用、用户界面、数据分析、数据统计——无论是什么问题,Python都有框架可以解决。最近,Python已经被数据科学家当作是筛选大型数据集的一个关键工具。

7.C

为什么C语言至今依然这么受欢迎?原因在于体积。 C语言不但小巧、速度快,而且功能非常强大。

8. SQL

SQL能让我们用一种快速、可重复和可靠的方式找到确切的信息。使用SQL,你就可以轻松地从大型、复杂的数据库中查询和提取有意义的数据。

9.Ruby

想要快速创建自己的项目吗?想要创建下一个大型web应用的思路原型吗?那么用Ruby(和Ruby on Rails)吧。 Ruby语言非常简单易学,其功能又令人难以置信的强大。

10. Objective-C

如果你有兴趣做一个适用于iOS的应用程序,那么你需要知道Objective-C。虽然去年的炒作主要集中在苹果的新的语言Swift上,但是Objective-C依然是构建苹果生态系统app的基础语言。

11. Perl

虽然Perl深奥又有点凌乱,但是我们不能否认的是它是一门功能超级强大的编程语言,也是任何人网络安全武器库的重要组成部分。Perl被认为是任何IT专业人士的关键工具。

12..NET

虽然其本身并不是语言,不过.NET是一个用于云服务和应用开发的微软平台。得利于微软近期的开源力度,.NET也成为了谷歌和苹果的平台。这样一来,你就可以使用.NET和多种编程语言轻松构建支持多个平台的app了。

13. Visual Basic

Visual Basic是.NET平台的一种关键语言,它既允许你构建app来支持你的业务,也可以自动化功能强大的Office应用,如Excel,还能简化最常见的任务。

14.R

R促进了大数据的革命,也是2015年凡是需要做认真数据分析的小伙伴都必须知道的编程语言。从科学和商业再到娱乐和社交媒体,R对几乎所有需要进行统计分析的领域都是好助手。

15.Swift

诞生至今甚至还不足一年,Swift作为一种方法和手段,一种可以快速便捷地开发苹果的Mac和iOS操作系统的方法和手段,迅速抓住了来自世界各 地的眼球,大量的开发人员趋之若鹜。Swift凭借它种类繁多的功能和友好的语法,使得任何人只需要一台Mac,就可以为iOS和Mac OS X打造下一个杀手级应用.

安装PIWIK

Standard

一直想给网站装一个监控点击量的应用。今天发现了PIWIK (http://piwik.org/),貌似不错,就决定装上去试试。

 

(1)首先当然是从PIWIK网站下载当前的版本;

(2)解压后里面有一个安装说明。基本上我就是按着这个说明做的;

(3)看了一下基本的配置要求(看说明的时候还有些担心不满足基本要求,后来装上后发现godaddy满足条件,没有问题)

  • Webserver such as Apache, Nginx, IIS, etc.
  • PHP version 5.3.3 or greater
  • MySQL version 4.1 or greater
  • (enabled by default) PHP extension pdoand pdo_mysql, or the mysqli extension.

Note that in May 2015 we will drop support for PHP 5.3 and Piwik will require PHP 5.4. Until then Piwik works well with any PHP >= 5.3.3.

 

(4)安装前要先在MySOL里面创建一个新的数据库,进入godaddy账户后,在manage host里面可以用相关的数据库应用MySQL phpMyAdmin创建; 创建好了之后,可以看到相关的信息(如数据库server的IP地址等),后面需要用到

 

(5)然后用Godaddy中的File Manager在根目录下创建一个目录(比如analytics)

(6)然后上传压缩的piwik.zip文件(上传压缩文件省时间,上传之后再解压)到该目录中,解压

(7)然后在浏览器中输入:http://yourdomain.com/analytics/piwik/index.php,就可以启动安装界面

(8)启动之后会首先检查系统是否满足要求(本人在安装时都满足,所以就到下一步,如果有问题,就需要解决)

(9)然后下一步中需要设置相关的数据库(这里就用到了前面创建的数据库),其中需要设置的项包括:database server, login, password,其它的没有改动;正常的话,会告诉Table created with success!

(10)然后,下一页中创建piwik的超级用户

(11)接下来一页中设置要监控的网站地址

(12)接下来的一页中会显示需要粘帖到网页中的代码(本人是在home page上加的,也在footer上加了; 当然也有插件可以直接用,如在Magento connect里的,但是我在电商网站上用,好像没成功,不知道什么原因)

(13)至此应该就算装好了

(14)登录http://mydomain.com/analytics/piwik/index.php,就可以看到结果了。

FW: 15款值得学习的小型开源项目,带你快速步入开源世界

Standard

http://code.csdn.net/news/2822568

对于初学者来说,小型开源项目更容易理解和学习,那么目前有哪些值得学习的小型开源项目?本文收集整理了知乎上网友推荐的一些不错的小型项目,代码基本都在5000行以内,感兴趣的朋友可以研究下~


SeaJS
Sea.js是一个遵循CommonJS规范的JavaScript模块加载框架,可以实现JavaScript的模块化开发及加载机制。Sea.js 追求简单、自然的代码书写和组织方式,代码非常精简。

Koa
Koa 是下一代的 Node.js 的 Web 框架。由 Express 团队设计。旨在提供一个更小型、更富有表现力、更可靠的 Web 应用和 API 的开发基础。

pssh
pssh 提供了并行版本的 OpenSSH 工具,特别适合用来控制有大量机器需要连接的情况,包括并行版本的 ssh、scp、rsync 和 kill 命令。该项目是Python写的,代码清晰而简短,数据结构定义的很清楚,看了之后你会称赞的。

grep-at-point 
在写代码的时候,有时候会需要查看某个函数/类/变量在哪些地方被使用过。该项目用emacs lisp语言写成,可实现在项目目录下,所有指定扩展名的文件中,搜索当前光标所在的单词。

compile-make
该项目可实现emacs中一键编译的需求。在项目目录下寻找Makefile文件,并执行make命令进行编译。 语言:emacs lisp ;代码行数:19。

sigslot
很精简的 signal slot的实现,跨平台。这个源码有2000多行,但是一大半是为了模板适配不同个数的参数的代码,干货不足1000行。   Gtalk源码剖析之:sigslot介绍

UCOS
uC/OS II(Micro Control Operation System Two)是一个可以基于 ROM运行的、可裁减的、抢占式、实时多任务内核,具有高度可移植性,特别适合于微处理器和控制器,是和很多商业操作系统性能相当的实时操作系统(RTOS)。

SQLite
SQLite是遵守ACID的关联式数据库管理系统,它包含在一个相对小的C库中。它是D.RichardHipp建立的公有领域项目。SQLite亦可以作为桌面数据库使用。

thttpd
thttpd 是一个小型的、简单的、可移植的、快速以及安全的HTTP服务器。

Boa Webserver
boa是一个小型的web服务器,可以用于多种平台,在嵌入式中比较常见。

C4
4个函数实现的c编译器,大约500行。基本上已经比较完备了,可以自己编译自己。

GLib
GLib 是一个包含很多有用的 C 程序的开发包,例如树、哈希、列表等。GLib 之前是属于 GTK 工具包的一部分,现在独立出来成为单独项目。

tj/co 
不到三百行,用于改善 node 异步回调语法。

shadowsocks
shadowsocks 是一个基于 python 的轻量级 socks 代理软件(谁用谁知道)。

aosabook
这是一本牛书aosabook的续集,500 Lines or Less的源码。里面每一章的代码不超过500行,实现了web服务器、爬虫、OCR等等“大型系统”,每一章由不同领域的大牛完成。看看作者列表,可以说是软件领域的名人堂。

100 Most Influential Books According to Stack Overflow

Standard

http://cspray.github.io/my.so-archive/100-most-influential-programming-books.html

100 Most Influential Books According to Stack Overflow

Please read this blog post to see why this is here. This data was created on 02/13/2012 20:00:00

All content found below was taken from Stack Overflow in accordance with the Creative Commons license governing the site. As such, you are free to use this data as you wish so long as the proper attribution is used.

Code Complete (2nd Ed) by Steve McConnell

cover

  • “The encyclopedia of good programming practice, Code Complete focuses on individual craftsmanship — all the things that add up to what we instinctively call “writing clean code.” This is the kind of book that has 50 pages just talking about code layout and whitespace.” —Joel (NB imo there’s more to it than semantics)

The Pragmatic Programmer; it’s more about your trade, and how to apply it than the code per se, but it’s still very good.

This is a great book for programmers who have learned the mechanics of programming, maybe in college, but don’t quite feel secure deciding what to do. It’s like the difference between drafting and architecture. What you learned in that class in college was drafting, and you can draw beautifully, but if you still feel like you wouldn’t quite know where to begin if someone told you to write a P2P music-swapping network all by yourself, this is the book for you.

Joel

The Pragmatic Programmer

enter image description here

Personally, Structure and Interpretation of Computer Programs is by far the most influential programming book I have ever read.

Some classics like Code Complete, Refactoring and Design Patterns teach you the effective working habits and the painstaking details of the trade. Others, like Peopleware, Psychology of Computer Programming and The Mythical Man-Month delve into the psychosocial aspects of software development. Numerous others deal with algorithms. These books all have their place.

SICP, however, is in a different league. It is a book that will enlighten you. It will evoke in you a passion for writing beautiful programs. Moreover, it will teach you to recognize and appreciate that very beauty. It will leave you with a state of awe and an unquenchable thirst to learn more. Other books may make you a better programmer; this book will make you a programmer.

And in the meanwhile, you will learn a thing or two about functional programming (side effects won’t be introduced until chapter three), lazy evaluation, metaprogramming (well, metalinguistic abstraction), virtual machines, interpreters, and compilers.

Some think that SICP is not a beginner’s book. Personally, I probably wouldn’t have appreciated the book in full without having some programming experience under my belt, but I would definitely recommend it for a beginner. The book is, after all, written for the famous 6.001, the introductory programming course at MIT. It may require an intellectual effort (especially if you do the exercises – and you should), but the reward is well worth the price.

Not convinced? Read the Foreword or the Preface to the First Edition. The full text is freely available on the web.

It’s available online for free, and there are even video lectures to go along with it.

The C Programming Language by Kernighan and Ritchie.

The C Programming Language Book

It is concise, easy to read, and it will teach you three things: the C programming language, how to think like a programmer, and the low-level computational model. (It is important to understand what’s going on “under the hood”.)

Introduction to algorithms (Cormen, Leiserson, Rivest, Stein) aka CLRS – Code Complete teaches you how to program correctly, The Mythical Man-Month teaches you how to manage correctly, Design Patterns teaches you how to design correctly…

This book teaches you how to write think about algorithms.

Introduction to algorithms cover image

In my mind, code is just a tool, not the essence. The main part of developing software (IMO) is creating new algorithms or re-implementing existing ones. Other than that it’s just assembling Lego bricks or creating “management” layers. I still dream about the workplace that would allow me to invest most (>50%) of my time in writing algorithms, leaving the management details to other people…

I personally think Design Patterns by the Gang of Four is a very useful book. It’s not about the “meta” aspects of programming like so many of the other suggestions, but it emphasises encapsulating good programming techniques as patterns, and has since encouraged others to come up with new patterns and antipatterns to use in programming dialogue.

Now for a rider….

@kevin, @modesty: Great answers! If I could place a 3-vote like on uservoice, I’d gladly use it here.

To the naysayers who downmodded them, I say: please, grow a sense of humour! :-)

I think I would have to recommend Refactoring: Improving the Design of Existing Code.

Refactoring: Improving the Design of Existing Code

In podcast 12, Jeff and Joel list a myriad of recommended books. Personally though I highly recommend The Mythical Man-Month.

image

The Art of Computer Programming if only for the effort Knuth put into it.

First Volume Hardcover

I’m surprised no one’s mentioned the Dragon Book by Aho et al. (or if it has been mentioned, I missed it).

Compilers (The Dragon Book) by Aho et al Newer Version

I will never forget the first edition’s cover. This book made me realize just how magically awesome compilers truly are. :)

I know that Design Patterns by the Gang of Four is a standard text, but rather than try and read that brick of a book start with the easier Head-First Design Patterns, and once you have got your head around the basic principles, progress to the great GoF bible…

For a truly deep read, I’d suggest Douglas Hofstadter’s Gödel, Escher, Bach. He dives pretty deep into many of the issues that programmers face every day- recursion, verifiability, proof, and boolean algebra. Great read, a little off the beaten path, occasionally challenging, and extremely rewarding once you fight through it and process what you’ve read.

Godel-Escher-Bach Cover

Clean Code

Clean Code has a lot in common with Code Complete but it’s more concise and practical with lots of clear examples.

Effective C++ and More Effective C++

From the early days of my career, Scott Meyer’s Effective C++ and later More Effective C++ both had an immediate impact on my programming ability. As a friend put it at the time, those books allow you to short cut the process of developing programming skills that otherwise would have taken years.

alt text

In the last year, the book with the biggest impact on my thinking has been The Cathedral and the Bazaar which taught me a lot about how the open source development process works and how to get rid of bugs from my code.

I recommend CODE by Charles Petzold. In this age of tools and IDEs that abstract a lot of complexity away from the programmers, this one is an eye opener.

book cover

I really recommend programming pearls, it’s got some amazing stuff in it, although I’m not ashamed to admit that I didn’t understand half of it!

alt text

Working Effectively with Legacy Code

by Michael Feathers. I don’t think that any book has affected my opinion of how I code more than this one. It explicitly tells you how to deal with someone elses code but implicitly you’ll learn what to avoid (and why).

Edit: Makes sense now.

alt text

For me the most influencal book is “Zen and the Art of Motorcycle Maintenance” by Robert Pirsig. It is all about no matter what you do, always strive for perfection, know your tools and task at hand inside-out, and, most of all, have fun (because if you are having fun, everything automatically leads to better results).

alt text

This book has a more recent edition (2000).

Two great long-quotes from it

Quality — you know what it is, yet you don’t know what it is. But that’s self- contradictory. But some things are better than others, that is, they have more quality. But when you try to say what the quality is, apart from the things that have it, it all goes poof! There’s nothing to talk about. But if you can’t say what Quality is, how do you know what it is, or how do you know that it even exists? If no one knows what it is, then for all practical purposes it doesn’t exist at all. But for all practical purposes it really does exist. What else are the grades based on? Why else would people pay fortunes for some things and throw others in the trash pile? Obviously some things are better than others — but what’s the “betterness”? — So round and round you go, spinning mental wheels and nowhere finding anyplace to get traction. What the hell is Quality? What is it?

 

Mountains should be climbed with as little effort as possible and without desire. The reality of your own nature should determine the speed. If you become restless, speed up. If you become winded, slow down. You climb the mountain in an equilibrium between restlessness and exhaustion. Then, when you’re no longer thinking ahead, each footstep isn’t just a means to an end but a unique event in itself. This leaf has jagged edges. This rock looks loose. From this place the snow is less visible, even though closer. These are things you should notice anyway. To live only for some future goal is shallow. It’s the sides of the mountain which sustain life, not the top. Here’s where things grow.

Get it here

Peopleware

Demarco and Lister demonstrate that the major issues of software development are human, not technical. Their answers aren’t easy–just incredibly successful. New second edition features eight all-new chapters.

alt text

Coders at Work by Peter Seibel. A very influential book to learn from the experience of some of the top names in the field, how they think and work.

alt text

ok, this is a slightly off-center answer, but believe it or not, it was on the reading list for a compsci course way back in the day. An excellent role model and a good book about curiosity.

Surely You’re Joking, Mr. Feynman!

Effective Java 2ed will teach you how to write beautiful and effective code. It’s a Java book, but there are many cross-language concepts.

Surprised that no one has mentioned Martin Fowler’s Patterns of Enterprise Application Architecture yet

alt text

I agree with Cristian, I think we should not forget SICP, I think every programmer should use it, al least as an exercise, you can complement it with;

The Little Schemer
alt text and

The Seasoned Schemer

alt text

I also include in the reading list Code Complete, Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and the Unified Process and Design Patterns and Design Patterns: Elements of Reusable Object-Oriented Software

book cover

The Inmates Are Running The Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity, or any of Alan Cooper’s books.

Because most programmers produce more WTFs/minute in the user interface than they do in the source code.

@lurks

The Art Of UNIX Programming by Eric S. Raymond

It is useful regardless operating system you use.

Many of the books already mentioned opened my eyes and influenced me, but a book every programmer should read is Test-Driven Development by Example. It really showed me the importance of unit tests and TDD and got me started very quick.

cover

Practices of an Agile Developer

Working in the Real World.

Practices of an Agile Developer Book Cover

Strunk, William. Elements of Style

elements of style

It depends on exactly what purpose you’re aiming for – I like Code Complete for pure programming, and Don’t Make Me Think is a great book on UI design.

enter image description here

How to Win Friends and Influence People by Dale Carnegie.

It taught me that in order to persuade people of your ideas, or lead them away from bad practices, you can never tell them that they are wrong. To do so will only entrench them further in their own ideas.

Domain Driven Design by Eric Evans

cover

alt text

I’d suggest “Modern C++ Design” by Andrei Alexandrescu, a really astonishing book about the awesome tricks and patterns you can achieve with C++, preprocessor directives and templates.

Modern C++ Design

I have a different answer — I really liked Joel’s Best Software Writing I.

Maybe that’s just me… but that collection opened my eyes to the “bigger picture” and inspired me to think of my programming as an art/craft.

I found The Practice of Programming by Kernighan and Pike is a very good read.

A great follow on to Code Complete – indispensable once you start working on projects of any decent size & need to communicate to your stakeholders about project delivery dates, etc.

Software Estimation: Demystifying the Black Art by Steve McConnel

The Passionate Programmer cover

Not the most influential, but certainly a good read and absent so far is My Job Went To India by Chad Fowler. It consists of 52 chapters/mini essays giving guidance on how to differentiate yourself as a developer rather than a code monkey (whose work could easily be outsourced).

Edit by different person: This is now called “The Passionate Programmer“.

I’m going to go a different route than the other answers so far…

Hackers: Heroes of the Computer Revolution. It’s an informal history of computers that really gives you an amazing feel for how this whole “computer culture” formed. It had a very powerful effect on me when I read it, oh, sometime around 1988.

enter image description here

It’s an oldie, but still worth reading: Algorithms + Data Structures = Programs

alt text

Dilbert’s Guide To The Rest Of Your Life: Dispatches from Cubicleland

alt text

This is more truth than you’d think.

I recommend Writing Solid Code. Old, but still very much worth a read.

I’m amazed that no one has suggested The Story About Ping yet.

If this text is displayed then the link to the image may be broken - apologies to Lynx users.

alt text

Excellent book on getting up to speed with JavaScript, highly reccommend.

JavaScript – The Good Parts

Getting Real by 37 Signals. It doesn’t matter if you don’t “ship” something at the end of the day.

The books listed here in this question are great. Code Complete, Pragmatic Programmer, Head First Design Patterns, all great.

My contribution to this list is a free read and is really focused on getting back to (best practice) basics. Foundations of Programming is a free eBook from one of the contributors to the popular Alt.Net blogs CodeBetter, Karl Seguin.

Covers Domain Driven Design, Persistence, Dependency Injection, Unit Testing, ORM etc.

alt text

I know this is a graphics book, but I am a graphics programmer and have been my whole career. Plus it’s written in ‘C’ :)

It’s as fundamental to me as the original K&R C Programming Language book.

Computer Graphics: Principles and Practice in C (2nd Edition) alt text

My favorite books are already covered here, but if you need to learn Java, I enjoyed Bruce Eckel’s book, Thinking in Java.

alt text

Code Complete is the classic.

However, Facts and Fallacies of Software Engineering by Robert Glass has a lot of good information on topics other than coding, for example people-issues, testing and process.

cover

The Elements Of Computing Systems

This book walks the reader through the process of building a computer system given NAND gates and flip flops. It gives a good introduction to the “big picture”.

The Elements Of Computing Systems

Refactoring to Patterns by Joshua Kerievsky

alt text

The Design of Everyday Things

and Things that Make Us Smart – both by Donald Norman

These apply to so much more than just user interface design… Make things that work as others would expect – even if the others are developers using code that you’ve created.

The Timeless Way of Building – Christopher Alexander

The original “patterns” book. Helps to understand why some software design just “feels” right and some does not.

The Annotated Turing was enlightening. It defines the box programmers work in.

alt text

alt text

Not the most important, but a very fundamental one

This book really set up a fundemental view that a programmer need to know about computer.

Computer System – A programmer’s perspective

alt text

I prefer Stroustrup’s The C++ Programming Language (3rd edition) – it’s the C++ book. C++ may be out of fashion these days, but this book takes you from the basics to OO to templates (STL) and even covers things like improving compile and link time. I still learn something every time I pick it up, and it’s never, ever bad to know C++.

My career as a developper is starting, so I cannot directly answer the question. But as .NET developper, the book that helped me the most in framework and reusable libraries design is “Framework Design Guidelines” by Brad Abrams. The book solves a lot of issue you can encounter when you are thinking about naming and crucial design choices. This is the best way to produce high reusable assemblies in .NET.

Framework Design Guidelines

I am pretty surprised that nobody mentioned that book. What is sure is that it has been really influential on my design and naming habits when writing reusable assemblies.

One thing I really appreciate in this book is the notes of .NET Framework developers giving details on issues they had when designing the .NET Framework, which I think are good advices as it is representative of a well designed framework.

It’s a brand new book and I’ve only just skimmed it but Growing Object-Oriented Software, Guided by Tests right now is THE book on TDD. It really concentrates on the “design” part of software development and how it is closely related to how you develop code writing tests first. I’ve enjoyed other books on TDD and unit testing but this takes it to a higher level, I’m pretty sure this one is destined to be a classic.

Growing Object-Oriented Software, Guided by Tests

I keep these 3 books on my shelf. As a reminder of the complex and exciting craft we do. I only recommend to read them if you happen to encounter with something that not even the entire Stackoverflow comunity can help you solve as I have:

alt text alt text alt text

=)

Alice’s Adventures in Wonderland Best programmers have read the book!

Kernighan and Ritchie’s “The C Programming Language” – The only C book you’ll ever need.

There are alot of really great books in the answers. One that isn’t mentioned that I absolutely love is Object Thinking by Dr. David West. This book had a huge impact on me because it explains the why more than the how.

Object Thinking

I’m really surprised that nobody has mentioned Advanced Programming in the UNIX Environment by W. Richard Stevens. It’s not necessarily the book that had the most impact on me, but it definitely deserves a place among many of the other greats mentioned here.

Even though I mostly did (and do) Perl programming, reading this book really helped me understand more about what’s going on under the hood. It covers a ton of really critical low-level concepts like File IO, system files (passwd, group, etc), process control, signals, and so on.

Having some idea of how this stuff works at the C level is very useful, even if you never write any C code, because every language you use (on a Unix system) is using these APIs under the hood.

Godel Escher Bach wins hands down for me, but that’s already been mentioned elsewhere. One book that motivated me highly that isn’t already in an answer AFAIK is Tracy Kidder’s Pulitzer prize winning book The Soul of a New Machine (amazon). Which charts the trials and tribulations of an engineering team trying to bring a new machine to market. It made me feel proud to be an engineer.

The soul of a New Machine

Hackers and Painters: Big Ideas from the Computer Age

CLR via C#, by Jeffrey Richter

alt text

Christopher Alexander – The Timeless Way of Building

Framework Design Guidelines will give you a very good start on how to organize code.

I thought Design Patterns in C# by Steve Metsker was good. The examples are a bit more advanced (and useful) than some other design pattern books I’ve read.

alt text

Since you didn’t say you wanted programming books specifically, I can be a little more creative.

Every programmer should have read Lewis Carol’s Alice in Wonderland.

Slightly more down-to-earth – every programmer should read Getting Things Done – especially the ones that I work with, because once you ‘get it’ you will be annoyed at how unorganized other people can be :)

I though Jurgen did a decent job with his Top 100 Best Software Engineering Books, Ever.

Maybe not the single most influential for every programmer but… to expand really expand your mind about computation in general, and to learn to write some very interesting programs, I recommend the
Computational Beauty of Nature.

alt text

BTW, the author is now head of Microsoft’s Live Labs.

Robert M. Pirsig’s Zen and the Art of Motorcycle Maintenance has a little section near the end about Gumption Traps. That’s the best advice I’ve ever read on how to debug code or solve problems in general. The rest of the book is pretty good, too.

About Face – The Essentials of Interaction Design (now in third edition)

This book opened my eyes to a very different viewpoint of development, focused on the interactions of the end user. It also helped me realize that creating great software is about more than just patterns and architecture, it’s about helping people achieve their goals.

Writing Solid Code, by Steve Maguire. Code Complete is a close runner-up.

Here Comes Everybody: The Power of Organizing Without Organizations – Clay Shirky alt text

This is an incredible book about the social effects of the internet. A must read for anyone in the tech industry, doubly so for programmers.

Here Comes Everybody: The Power of Organizing Without Organizations is a book about what happens when people are given the tools to do things together, without needing traditional organizational structures.

I was lucky enough to read this pretty early in my so-called career:

Philip and Alex’s Guide to Web Publishing

It was cutting edge in 1998 and still has plenty of relevant points. I found it an enjoyable read with a real sense of humour (not the twee kind you often get in software books). When he gets down to the details he talks about specific platforms though, so it shows it’s age when Oracle 7 is mentioned!

I would still put this at the top of the required reading list for a web developer because of the way it assumes no prior knowledge, starting from first principles (“what’s HTML?”)

alt text

JavaScript: The Good Parts by Douglas Crockford

Opened my eyes to the goodness hidden in JavaScript and how bad the examples of JavaScript are strewn through tutorial sites.

Essential reading for anyone serious about web client side development.

It also improved my C# and showed me the light to dynamic languages.

This is a great book for a Java developer new or old:

Effective Java by Joshua Bloch

There are similar questions here and here.

I don’t know if it were already suggested but:

alt text

The Tao Te Ching. Although it’s a philosophy book and not a programming book, some of it’s principles are very applicable to programming:

“Know when it’s time to stop. If you don’t know then stop when you are done.”

Translation: Knowing your requirements means you know when to stop. If you don’t know when to stop, you need to stop because the requirements have yet to be defined.

“The harder one tries, the more resistance one creates for oneself.”

Translation: How many times have you worked on a problem for several hours, only to find the answer after taking a 15-minute break? The more you hammer at a problem, the harder it is going to be for you to solve it.

“One whose needs are simple can fulfill them easily.”

Translation: Simple requirements lead to simple designs.

“When we lose the fundamentals, we supplant them with increasingly inferior values which we pretend are the true values.”

Translation: Hubris is never a good substitute for good programming standards. If you get lax, no amount of design patterns will ever substitute for the lack of quality in your code.

Most fun to read: The Art of Deception by Kevin Mitnick (one of world’s most famous hackers)

Really shows that how secure your software may be, the weakest link is always the human controlling it, with entertaining examples.

alt text

The Productive programmer. its a natural extension of “the pragmatic programer” which gets a little more into the every day details.

alt text

If you could go back in time and tell yourself to read a specific book at the beginning of your career as a developer, which book would it be?

JUnit in Action

Unit testing is one thing that I really wished I knew at the start of my programming career rather than the middle of it.

As to the general “what good books have you read” offshoot of this thread:

Mastering the Requirements Process

Writing requirements is one area where I suck, this book helped a lot.

Now off to order a book or two from Amazon… 😉

 

(FW)程序员整理的 C++ 资源大全

Standard

https://github.com/fffaraz/awesome-cpp

http://blog.jobbole.com/78901/

 

关于 C++ 框架、库和资源的一些汇总列表,由 fffaraz 发起和维护。

内容包括:标准库、Web应用框架、人工智能、数据库、图片处理、机器学习、日志、代码分析等。

 

标准库

C++标准库,包括了STL容器,算法和函数等。

 

框架

C++通用框架和库

  • Apache C++ Standard Library:是一系列算法,容器,迭代器和其他基本组件的集合
  • ASL :Adobe源代码库提供了同行的评审和可移植的C++源代码库。
  • Boost :大量通用C++库的集合。
  • BDE :来自于彭博资讯实验室的开发环境。
  • Cinder:提供专业品质创造性编码的开源开发社区。
  • Cxxomfort:轻量级的,只包含头文件的库,将C++ 11的一些新特性移植到C++03中。
  • Dlib:使用契约式编程和现代C++科技设计的通用的跨平台的C++库。
  • EASTL :EA-STL公共部分
  • ffead-cpp :企业应用程序开发框架
  • Folly:由Facebook开发和使用的开源C++库
  • JUCE :包罗万象的C++类库,用于开发跨平台软件
  • libPhenom:用于构建高性能和高度可扩展性系统的事件框架。
  • LibSourcey :用于实时的视频流和高性能网络应用程序的C++11 evented IO
  • LibU : C语言写的多平台工具库
  • Loki :C++库的设计,包括常见的设计模式和习语的实现。
  • MiLi :只含头文件的小型C++库
  • openFrameworks :开发C++工具包,用于创意性编码。
  • Qt :跨平台的应用程序和用户界面框架
  • Reason :跨平台的框架,使开发者能够更容易地使用Java,.Net和Python,同时也满足了他们对C++性能和优势的需求。
  • ROOT :具备所有功能的一系列面向对象的框架,能够非常高效地处理和分析大量的数据,为欧洲原子能研究机构所用。
  • STLport:是STL具有代表性的版本
  • STXXL:用于额外的大型数据集的标准模板库。
  • Ultimate++ :C++跨平台快速应用程序开发框架
  • Windows Template Library:用于开发Windows应用程序和UI组件的C++库
  • Yomm11 :C++11的开放multi-methods.

 

人工智能

  • btsk :游戏行为树启动器工具
  • Evolving Objects:基于模板的,ANSI C++演化计算库,能够帮助你非常快速地编写出自己的随机优化算法。
  • Neu:C++11框架,编程语言集,用于创建人工智能应用程序的多用途软件系统。

 

异步事件循环

  • Boost.Asio:用于网络和底层I/O编程的跨平台的C++库。
  • libev :功能齐全,高性能的时间循环,轻微地仿效libevent,但是不再像libevent一样有局限性,也修复了它的一些bug。
  • libevent :事件通知库
  • libuv :跨平台异步I/O。

 

音频

音频,声音,音乐,数字化音乐库

  • FMOD :易于使用的跨平台的音频引擎和音频内容的游戏创作工具。
  • Maximilian :C++音频和音乐数字信号处理库
  • OpenAL :开源音频库—跨平台的音频API
  • Opus:一个完全开放的,免版税的,高度通用的音频编解码器
  • Speex:免费编解码器,为Opus所废弃
  • Tonic: C++易用和高效的音频合成
  • Vorbis: Ogg Vorbis是一种完全开放的,非专有的,免版税的通用压缩音频格式。

 

生态学

生物信息,基因组学和生物技术

  • libsequence:用于表示和分析群体遗传学数据的C++库。
  • SeqAn:专注于生物数据序列分析的算法和数据结构。
  • Vcflib :用于解析和处理VCF文件的C++库
  • Wham:直接把联想测试应用到BAM文件的基因结构变异。

压缩

压缩和归档库

  • bzip2:一个完全免费,免费专利和高质量的数据压缩
  • doboz:能够快速解压缩的压缩库
  • PhysicsFS:对各种归档提供抽象访问的库,主要用于视频游戏,设计灵感部分来自于Quake3的文件子系统。
  • KArchive:用于创建,读写和操作文件档案(例如zip和 tar)的库,它通过QIODevice的一系列子类,使用gzip格式,提供了透明的压缩和解压缩的数据。
  • LZ4 :非常快速的压缩算法
  • LZHAM :无损压缩数据库,压缩比率跟LZMA接近,但是解压缩速度却要快得多。
  • LZMA :7z格式默认和通用的压缩方法。
  • LZMAT :及其快速的实时无损数据压缩库
  • miniz:单一的C源文件,紧缩/膨胀压缩库,使用zlib兼容API,ZIP归档读写,PNG写方式。
  • Minizip:Zlib最新bug修复,支持PKWARE磁盘跨越,AES加密和IO缓冲。
  • Snappy :快速压缩和解压缩
  • ZLib :非常紧凑的数据流压缩库
  • ZZIPlib:提供ZIP归档的读权限。

 

并发性

并发执行和多线程

  • Boost.Compute :用于OpenCL的C++GPU计算库
  • Bolt :针对GPU进行优化的C++模板库
  • C++React :用于C++11的反应性编程库
  • Intel TBB :Intel线程构件块
  • Libclsph:基于OpenCL的GPU加速SPH流体仿真库
  • OpenCL :并行编程的异构系统的开放标准
  • OpenMP:OpenMP API
  • Thrust :类似于C++标准模板库的并行算法库
  • HPX :用于任何规模的并行和分布式应用程序的通用C++运行时系统
  • VexCL :用于OpenCL/CUDA 的C++向量表达式模板库。

 

容器

  • C++ B-tree :基于B树数据结构,实现命令内存容器的模板库
  • Hashmaps: C++中开放寻址哈希表算法的实现

 

密码学

  • Bcrypt :一个跨平台的文件加密工具,加密文件可以移植到所有可支持的操作系统和处理器中。
  • BeeCrypt
  • Botan: C++加密库
  • Crypto++:一个有关加密方案的免费的C++库
  • GnuPG: OpenPGP标准的完整实现
  • GnuTLS :实现了SSL,TLS和DTLS协议的安全通信库
  • Libgcrypt
  • libmcrypt
  • LibreSSL:免费的SSL/TLS协议,属于2014 OpenSSL的一个分支
  • LibTomCrypt:一个非常全面的,模块化的,可移植的加密工具
  • libsodium:基于NaCI的加密库,固执己见,容易使用
  • Nettle 底层的加密库
  • OpenSSL : 一个强大的,商用的,功能齐全的,开放源代码的加密库。
  • Tiny AES128 in C :用C实现的一个小巧,可移植的实现了AES128ESB的加密算法

 

数据库

数据库,SQL服务器,ODBC驱动程序和工具

  • hiberlite :用于Sqlite3的C++对象关系映射
  • Hiredis: 用于Redis数据库的很简单的C客户端库
  • LevelDB: 快速键值存储库
  • LMDB:符合数据库四大基本元素的嵌入键值存储
  • MySQL++:封装了MySql的C API的C++ 包装器
  • RocksDB:来自Facebook的嵌入键值的快速存储
  • SQLite:一个完全嵌入式的,功能齐全的关系数据库,只有几百KB,可以正确包含到你的项目中。

 

调试

调试库, 内存和资源泄露检测,单元测试

  • Boost.Test:Boost测试库
  • Catch:一个很时尚的,C++原生的框架,只包含头文件,用于单元测试,测试驱动开发和行为驱动开发。
  • CppUnit:由JUnit移植过来的C++测试框架
  • CTest:CMake测试驱动程序
  • googletest:谷歌C++测试框架
  • ig-debugheap:用于跟踪内存错误的多平台调试堆
  • libtap:用C语言编写测试
  • MemTrack —用于C++跟踪内存分配
  • microprofile– 跨平台的网络试图分析器
  • minUnit :使用C写的迷你单元测试框架,只使用了两个宏
  • Remotery:用于web视图的单一C文件分析器
  • UnitTest++:轻量级的C++单元测试框架

 

游戏引擎

  • Cocos2d-x :一个跨平台框架,用于构建2D游戏,互动图书,演示和其他图形应用程序。
  • Grit :社区项目,用于构建一个免费的游戏引擎,实现开放的世界3D游戏。
  • Irrlicht :C++语言编写的开源高性能的实时#D引擎
  • Polycode:C++实现的用于创建游戏的开源框架(与Lua绑定)。

 

图形用户界面

  • CEGUI : 很灵活的跨平台GUI库
  • FLTK :快速,轻量级的跨平台的C++GUI工具包。
  • GTK+: 用于创建图形用户界面的跨平台工具包
  • gtkmm :用于受欢迎的GUI库GTK+的官方C++接口。
  • imgui:拥有最小依赖关系的立即模式图形用户界面
  • libRocketlibRocket 是一个C++ HTML/CSS 游戏接口中间件
  • MyGUI :快速,灵活,简单的GUI
  • Ncurses:终端用户界面
  • QCustomPlot :没有更多依赖关系的Qt绘图控件
  • Qwt :用户与技术应用的Qt 控件
  • QwtPlot3D :功能丰富的基于Qt/OpenGL的C++编程库,本质上提供了一群3D控件
  • OtterUIOtterUI 是用于嵌入式系统和互动娱乐软件的用户界面开发解决方案
  • PDCurses 包含源代码和预编译库的公共图形函数库
  • wxWidgets C++库,允许开发人员使用一个代码库可以为widows, Mac OS X,Linux和其他平台创建应用程序

 

图形

  • bgfx:跨平台的渲染库
  • Cairo:支持多种输出设备的2D图形库
  • Horde3D 一个小型的3D渲染和动画引擎
  • magnum C++11和OpenGL 2D/3D 图形引擎
  • Ogre 3D 用C++编写的一个面向场景,实时,灵活的3D渲染引擎(并非游戏引擎)
  • OpenSceneGraph 具有高性能的开源3D图形工具包
  • Panda3D 用于3D渲染和游戏开发的框架,用Python和C++编写。
  • Skia 用于绘制文字,图形和图像的完整的2D图形库
  • urho3d 跨平台的渲染和游戏引擎。

 

图像处理

  • Boost.GIL:通用图像库
  • CImg :用于图像处理的小型开源C++工具包
  • CxImage :用于加载,保存,显示和转换的图像处理和转换库,可以处理的图片格式包括 BMP, JPEG, GIF, PNG, TIFF, MNG, ICO, PCX, TGA, WMF, WBMP, JBG, J2K。
  • FreeImage :开源库,支持现在多媒体应用所需的通用图片格式和其他格式。
  • GDCM:Grassroots DICOM 库
  • ITK:跨平台的开源图像分析系统
  • Magick++:ImageMagick程序的C++接口
  • MagickWnd:ImageMagick程序的C++接口
  • OpenCV : 开源计算机视觉类库
  • tesseract-ocr:OCR引擎
  • VIGRA :用于图像分析通用C++计算机视觉库
  • VTK :用于3D计算机图形学,图像处理和可视化的开源免费软件系统。

 

国际化

  • gettext :GNU `gettext’
  • IBM ICU:提供Unicode 和全球化支持的C、C++ 和Java库
  • libiconv :用于不同字符编码之间的编码转换库

 

Jason

  • frozen : C/C++的Jason解析生成器
  • Jansson :进行编解码和处理Jason数据的C语言库
  • jbson :C++14中构建和迭代BSON data,和Json 文档的库
  • JeayeSON:非常健全的C++ JSON库,只包含头文件
  • JSON++ : C++ JSON 解析器
  • json-parser:用可移植的ANSI C编写的JSON解析器,占用内存非常少
  • json11 :一个迷你的C++11 JSON库
  • jute :非常简单的C++ JSON解析器
  • ibjson:C语言中的JSON解析和打印库,很容易和任何模型集成。
  • libjson:轻量级的JSON库
  • PicoJSON:C++中JSON解析序列化,只包含头文件
  • qt-json :用于JSON数据和 QVariant层次间的相互解析的简单类
  • QJson:将JSON数据映射到QVariant对象的基于Qt的库
  • RapidJSON: 用于C++的快速JSON 解析生成器,包含SAX和DOM两种风格的API
  • YAJL :C语言中快速流JSON解析库

 

日志

  • Boost.Log :设计非常模块化,并且具有扩展性
  • easyloggingpp:C++日志库,只包含单一的头文件。
  • Log4cpp :一系列C++类库,灵活添加日志到文件,系统日志,IDSA和其他地方。
  • templog:轻量级C++库,可以添加日志到你的C++应用程序中

 

机器学习

  • Caffe :快速的神经网络框架
  • CCV :以C语言为核心的现代计算机视觉库
  • mlpack :可扩展的C++机器学习库
  • OpenCV:开源计算机视觉库
  • Recommender:使用协同过滤进行产品推荐/建议的C语言库。
  • SHOGUN:Shogun 机器学习工具
  • sofia-ml :用于机器学习的快速增量算法套件

 

数学

  • Armadillo :高质量的C++线性代数库,速度和易用性做到了很好的平衡。语法和MatlAB很相似
  • blaze:高性能的C++数学库,用于密集和稀疏算法。
  • ceres-solver :来自谷歌的C++库,用于建模和解决大型复杂非线性最小平方问题。
  • CGal: 高效,可靠的集合算法集合
  • cml :用于游戏和图形的免费C++数学库
  • Eigen :高级C++模板头文件库,包括线性代数,矩阵,向量操作,数值解决和其他相关的算法。
  • GMTL:数学图形模板库是一组广泛实现基本图形的工具。
  • GMP:用于个高精度计算的C/C++库,处理有符号整数,有理数和浮点数。

 

多媒体

  • GStreamer :构建媒体处理组件图形的库
  • LIVE555 Streaming Media :使用开放标准协议(RTP/RTCP, RTSP, SIP) 的多媒体流库
  • libVLC :libVLC (VLC SDK)媒体框架
  • QtAv:基于Qt和FFmpeg的多媒体播放框架,能够帮助你轻而易举地编写出一个播放器
  • SDL :简单直控媒体层
  • SFML :快速,简单的多媒体库

 

网络

  • ACE:C++面向对象网络变成工具包
  • Boost.Asio:用于网络和底层I/O编程的跨平台的C++库
  • Casablanca:C++ REST SDK
  • cpp-netlib:高级网络编程的开源库集合
  • Dyad.c:C语言的异步网络
  • libcurl :多协议文件传输库
  • Mongoose:非常轻量级的网络服务器
  • Muduo :用于Linux多线程服务器的C++非阻塞网络库
  • net_skeleton :C/C++的TCP 客户端/服务器库
  • nope.c :基于C语言的超轻型软件平台,用于可扩展的服务器端和网络应用。 对于C编程人员,可以考虑node.js
  • Onion :C语言HTTP服务器库,其设计为轻量级,易使用。
  • POCO:用于构建网络和基于互联网应用程序的C++类库,可以运行在桌面,服务器,移动和嵌入式系统。
  • RakNet:为游戏开发人员提供的跨平台的开源C++网络引擎。
  • Tuf o :用于Qt之上的C++构建的异步Web框架。
  • WebSocket++ :基于C++/Boost Aiso的websocket 客户端/服务器库
  • ZeroMQ :高速,模块化的异步通信库

 

物理学

动力学仿真引擎

  • Box2D:2D的游戏物理引擎。
  • Bullet :3D的游戏物理引擎。
  • Chipmunk :快速,轻量级的2D游戏物理库
  • LiquidFun:2D的游戏物理引擎
  • ODE :开放动力学引擎-开源,高性能库,模拟刚体动力学。
  • ofxBox2d:Box2D开源框架包装器。
  • Simbody :高性能C++多体动力学/物理库,模拟关节生物力学和机械系统,像车辆,机器人和人体骨骼。

 

机器人学

  • MOOS-IvP :一组开源C++模块,提供机器人平台的自主权,尤其是自主的海洋车辆。
  • MRPT:移动机器人编程工具包
  • PCL :点云库是一个独立的,大规模的开放项目,用于2D/3D图像和点云处理。
  • Robotics Library (RL): 一个独立的C++库,包括机器人动力学,运动规划和控制。
  • RobWork:一组C++库的集合,用于机器人系统的仿真和控制。
  • ROS :机器人操作系统,提供了一些库和工具帮助软件开发人员创建机器人应用程序。

 

科学计算

  • FFTW :用一维或者多维计算DFT的C语言库。
  • GSL:GNU科学库。

 

脚本

  • ChaiScript :用于C++的易于使用的嵌入式脚本语言。
  • Lua :用于配置文件和基本应用程序脚本的小型快速脚本引擎。
  • luacxx:用于创建Lua绑定的C++ 11 API
  • SWIG :一个可以让你的C++代码链接到JavaScript,Perl,PHP,Python,Tcl和Ruby的包装器/接口生成器
  • V7:嵌入式的JavaScript 引擎。
  • V8 :谷歌的快速JavaScript引擎,可以被嵌入到任何C++应用程序中。

 

序列化

  • Cap’n Proto :快速数据交换格式和RPC系统。
  • cereal :C++11 序列化库
  • FlatBuffers :内存高效的序列化库
  • MessagePack :C/C++的高效二进制序列化库,例如 JSON
  • protobuf :协议缓冲,谷歌的数据交换格式。
  • protobuf-c :C语言的协议缓冲实现
  • SimpleBinaryEncoding:用于低延迟应用程序的对二进制格式的应用程序信息的编码和解码。
  • Thrift :高效的跨语言IPC/RPC,用于C++,Java,Python,PHP,C#和其它多种语言中,最初由Twitter开发。

 

视频

  • libvpx :VP8/VP9编码解码SDK
  • FFmpeg :一个完整的,跨平台的解决方案,用于记录,转换视频和音频流。
  • libde265 :开放的h.265视频编解码器的实现。
  • OpenH264:开源H.364 编解码器。
  • Theora :免费开源的视频压缩格式。

 

虚拟机

  • CarpVM:C中有趣的VM,让我们一起来看看这个。
  • MicroPython :旨在实现单片机上Python3.x的实现
  • TinyVM:用纯粹的ANSI C编写的小型,快速,轻量级的虚拟机。

 

Web应用框架

  • Civetweb :提供易于使用,强大的,C/C++嵌入式Web服务器,带有可选的CGI,SSL和Lua支持。
  • CppCMS :免费高性能的Web开发框架(不是 CMS).
  • Crow :一个C++微型web框架(灵感来自于Python Flask)
  • Kore :使用C语言开发的用于web应用程序的超快速和灵活的web服务器/框架。
  • libOnion:轻量级的库,帮助你使用C编程语言创建web服务器。
  • QDjango:使用C++编写的,基于Qt库的web框架,试图效仿Django API,因此得此名。
  • Wt :开发Web应用的C++库。

 

XML

XML就是个垃圾,xml的解析很烦人,对于计算机它也是个灾难。这种糟糕的东西完全没有存在的理由了。-Linus Torvalds

  • Expat :用C语言编写的xml解析库
  • Libxml2 :Gnome的xml C解析器和工具包
  • libxml++ :C++的xml解析器
  • PugiXML :用于C++的,支持XPath的轻量级,简单快速的XML解析器。
  • RapidXml :试图创建最快速的XML解析器,同时保持易用性,可移植性和合理的W3C兼容性。
  • TinyXML :简单小型的C++XML解析器,可以很容易地集成到其它项目中。
  • TinyXML2:简单快速的C++CML解析器,可以很容易集成到其它项目中。
  • TinyXML++:TinyXML的一个全新的接口,使用了C++的许多许多优势,模板,异常和更好的异常处理。
  • Xerces-C++ :用可移植的C++的子集编写的XML验证解析器。

 

多项混杂

一些有用的库或者工具,但是不适合上面的分类,或者还没有分类。

  • C++ Format :C++的小型,安全和快速格式化库
  • casacore :从aips++ 派生的一系列C++核心库
  • cxx-prettyprint:用于C++容器的打印库
  • DynaPDF :易于使用的PDF生成库
  • gcc-poison :帮助开发人员禁止应用程序中的不安全的C/C++函数的简单的头文件。
  • googlemock:编写和使用C++模拟类的库
  • HTTP Parser :C的http请求/响应解析器
  • libcpuid :用于x86 CPU检测盒特征提取的小型C库
  • libevil :许可证管理器
  • libusb:允许移动访问USB设备的通用USB库
  • PCRE:正则表达式C库,灵感来自于Perl中正则表达式的功能。
  • Remote Call Framework :C++的进程间通信框架。
  • Scintilla :开源的代码编辑控件
  • Serial Communication Library :C++语言编写的跨平台,串口库。
  • SDS:C的简单动态字符串库
  • SLDR :超轻的DNS解析器
  • SLRE: 超轻的正则表达式库
  • Stage :移动机器人模拟器
  • VarTypes:C++/Qt4功能丰富,面向对象的管理变量的框架。
  • ZBar:‘条形码扫描器’库,可以扫描照片,图片和视频流中的条形码,并返回结果。
  • CppVerbalExpressions :易于使用的C++正则表达式
  • QtVerbalExpressions:基于C++ VerbalExpressions 库的Qt库
  • PHP-CPP:使用C++来构建PHP扩展的库
  • Better String :C的另一个字符串库,功能更丰富,但是没有缓冲溢出问题,还包含了一个C++包装器。

 

软件

用于创建开发环境的软件

编译器

C/C++编译器列表

  • Clang :由苹果公司开发的
  • GCC:GNU编译器集合
  • Intel C++ Compiler :由英特尔公司开发
  • LLVM :模块化和可重用编译器和工具链技术的集合
  • Microsoft Visual C++ :MSVC,由微软公司开发
  • Open WatCom :Watcom,C,C++和Fortran交叉编译器和工具
  • TCC :轻量级的C语言编译器

 

在线编译器

在线C/C++编译器列表

  • codepad :在线编译器/解释器,一个简单的协作工具
  • CodeTwist:一个简单的在线编译器/解释器,你可以粘贴的C,C++或者Java代码,在线执行并查看结果
  • coliru :在线编译器/shell, 支持各种C++编译器
  • Compiler Explorer:交互式编译器,可以进行汇编输出
  • CompileOnline:Linux上在线编译和执行C++程序
  • Ideone :一个在线编译器和调试工具,允许你在线编译源代码并执行,支持60多种编程语言。

 

调试器

C/C++调试器列表

 

集成开发环境(IDE)

C/C++集成开发环境列表

  • AppCode :构建与JetBrains’ IntelliJ IDEA 平台上的用于Objective-C,C,C++,Java和Java开发的集成开发环境
  • CLion:来自JetBrains的跨平台的C/C++的集成开发环境
  • Code::Blocks :免费C,C++和Fortran的集成开发环境
  • CodeLite :另一个跨平台的免费的C/C++集成开发环境
  • Dev-C++:可移植的C/C++/C++11集成开发环境
  • Eclipse CDT:基于Eclipse平台的功能齐全的C和C++集成开发环境
  • Geany :轻量级的快速,跨平台的集成开发环境。
  • IBM VisualAge :来自IBM的家庭计算机集成开发环境。
  • Irony-mode:由libclang驱动的用于Emacs的C/C++微模式
  • KDevelop:免费开源集成开发环境
  • Microsoft Visual Studio :来自微软的集成开发环境
  • NetBeans :主要用于Java开发的的集成开发环境,也支持其他语言,尤其是PHP,C/C++和HTML5。
  • Qt Creator:跨平台的C++,Javascript和QML集成开发环境,也是Qt SDK的一部分。
  • rtags:C/C++的客户端服务器索引,用于 跟基于clang的emacs的集成
  • Xcode :由苹果公司开发
  • YouCompleteMe:一个用于Vim的根据你敲的代码快速模糊搜索并进行代码补全的引擎。

 

构建系统

  • Bear :用于为clang工具生成编译数据库的工具
  • Biicode:基于文件的简单依赖管理器。
  • CMake :跨平台的免费开源软件用于管理软件使用独立编译的方法进行构建的过程。
  • CPM:基于CMake和Git的C++包管理器
  • FASTBuild:高性能,开源的构建系统,支持高度可扩展性的编译,缓冲和网络分布。
  • Ninja :专注于速度的小型构建系统
  • Scons :使用Python scipt 配置的软件构建工具
  • tundra :高性能的代码构建系统,甚至对于非常大型的软件项目,也能提供最好的增量构建次数。
  • tup:基于文件的构建系统,用于后台监控变化的文件。

 

静态代码分析

提高质量,减少瑕疵的代码分析工具列表