FastDFS分布式文件系统的安装及配置
由于网站使用nfs共享方式保存用户上传的图片，附件等资料，然后通过apache下载的方式供用户访问，在网站架构初期，使用这种简单的方式实现了静态资源的读写分离，但随着网站数据量的增加，图片服务器渐渐成为整个网站的短板，缘次催生了使用fastfds的想法，故而先进行一番简单的测试！在开始之前还是先来看看fastfds的介绍信息：
|
FastDFS is an open source high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balancing. FastDFS is an open source high performance distributed file system. It's major functions include: file storing, file syncing and file accessing (file uploading and file downloading), and it can resolve the high capacity and load balancing problem. FastDFS should meet the requirement of the website whose service based on files such as photo sharing site and vidio sharing site. FastDFS has two roles: tracker and storage. The tracker takes charge of scheduling and load balancing for file access. The storage store files and it's function is file management including: file storing, file syncing, providing file access interface. It also manage the meta data which are attributes representing as key value pair of the file. For example: width=1024, the key is "width" and the value is "1024". The tracker and storage contain one or more servers. The servers in the tracker or storage cluster can be added to or removed from the cluster by any time without affecting the online services. The servers in the tracker cluster are peer to peer. The storarge servers organizing by the file volume/group to obtain high capacity. The storage system contains one or more volumes whose files are independent among these volumes. The capacity of the whole storage system equals to the sum of all volumes' capacity. A file volume contains one or more storage servers whose files are same among these servers. The servers in a file volume backup each other, and all these servers are load balancing. When adding a storage server to a volume, files already existing in this volume are replicated to this new server automatically, and when this replication done, system will switch this server online to providing storage services. When the whole storage capacity is insufficiency, you can add one or more volumes to expand the storage capacity. To do this, you need to add one or more storage servers. The identification of a file is composed of two parts: the volume name and the file name. |
大意为：
fastdfs是一个开源的，高性能的的分布式文件系统，他主要的功能包括：文件存储，同步和访问，设计基于高可用和负载均衡，fastfd非常适用于基于文件服务的站点，例如图片分享和视频分享网站
fastfds有两个角色：跟踪服务和存储服务，跟踪服务控制，调度文件以负载均衡的方式访问；存储服务包括：文件存储，文件同步，提供文件访问接口，同时以key value的方式管理文件的元数据
跟踪和存储服务可以由1台或者多台服务器组成，同时可以动态的添加，删除跟踪和存储服务而不会对在线的服务产生影响，在集群中，tracker服务是对等的
存储系统由一个或多个卷组成，卷与卷之间的文件是相互独立的，所有卷的文件容量累加就是整个存储系统中的文件容量。一个卷可以由一台或多台存储服务器组成，一个卷下的存储服务器中的文件都是相同的，卷中的多台存储服务器起到了冗余备份和负载均衡的作用。在卷中增加服务器时，同步已有的文件由系统自动完成，同步完成后，系统自动将新增服务器切换到线上提供服务。当存储空间不足或即将耗尽时，可以动态添加卷。只需要增加一台或多台服务器，并将它们配置为一个新的卷，这样就扩大了存储系统的容量。
下面几张图可以清楚的说明fastfds的架构和文件上传和下载流程等：
下面将介绍下fastdfs在rhel上的部署过程
tracker服务器：192.168.123.110/24
storage服务器：192.168.123.20/24
一：编译安装
[root@db1 ~]# wget http://cloud.github.com/downloads/libevent/libevent/libevent-2.0.14
-stable.tar.gz
[root@db1 ~]# tar -zxvpf libevent-2.0.14-stable.tar.gz
[root@db1 ~]# cd libevent-2.0.14-stable
[root@db1 libevent-2.0.14-stable]# ./configure --prefix=/usr/local/libevent-2.0.14 &&
make && make install
[root@db1 ~]# wget http://fastdfs.googlecode.com/files/FastDFS_v3.02.tar.gz
[root@db1 ~]# tar -zxvf FastDFS_v3.02.tar.gz
[root@db1 ~]# cd FastDFS
[root@db1 FastDFS]# vim make.sh 将#WITH_HTTPD=1修改成WITH_HTTPD=1以支持 http
[root@db1 FastDFS]# ./make.sh
[root@db1 FastDFS]# ./make.sh install
[root@db1 FastDFS]# ls /etc/fdfs/
client.conf http.conf mime.types storage.conf tracker.conf
二：tracker配置文件
[root@db1 ~]# grep -v '^#' /etc/fdfs/tracker.conf |grep -v '^$'
disabled=false
bind_addr=192.168.123.110
port=22122
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
max_connections=256
work_threads=4
store_lookup=2
store_group=group2
store_server=0
store_path=0
download_server=0
reserved_storage_space = 4GB
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
http.disabled=false
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
http.need_find_content_type=true
[root@db1 ~]# grep -v '^#' /etc/fdfs/http.conf |grep -v '^$'
http.default_content_type = application/octet-stream
http.mime_types_filename=/etc/fdfs/mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg
三：启动tracker服务，需要注意tracker.conf文件最后一行为#include httpd.conf
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
/usr/local/FastDFS/bin/fdfs_trackerd: error while loading shared libraries: libevent-
2.0.so.5: cannot open shared object file: No such file or directory
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf
[root@db1 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf
[root@db1 ~]# ldconfig
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[2012-07-04 17:52:25] ERROR - file: tracker_func.c, line: 160, "/home/data/fastdfs"
can't be accessed, error info: No such file or directory
[root@db1 ~]# mkdir -p /home/data/fastdfs
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@db1 ~]# echo $?
22
[root@db1 ~]# cat /home/data/fastdfs/logs/trackerd.log
[2012-07-04 17:52:50] ERROR - file: ../common/fdfs_http_shared.c, line: 128, param
"http.mime_types_filename" not exist or is empty
[root@db1 ~]# tail -1 /etc/fdfs/tracker.conf
#include http.conf
[root@db1 ~]# /usr/local/FastDFS/bin/fdfs_trackerd /etc/fdfs/tracker.conf
[root@db1 ~]# echo $?
0
[root@db1 ~]# ps -ef |grep track
root 3535 1 0 15:47 ? 00:00:00 /usr/local/FastDFS/bin/fdfs_trackerd
/etc/fdfs/tracker.conf
[root@db1 ~]# netstat -ntpl |grep fdfs
tcp 0 0 192.168.123.110:22122 0.0.0.0:* LISTEN
3535/fdfs_trackerd
tcp 0 0 192.168.123.110:8080 0.0.0.0:* LISTEN
3535/fdfs_trackerd
四：storage配置文件
[root@db2 ~]# grep -v '^#' /etc/fdfs/storage.conf |grep -v '^$'
disabled=false
group_name=group1
bind_addr=192.168.123.20
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/home/data/fastdfs
max_connections=256
buff_size = 256KB
work_threads=4
disk_rw_separated = true
disk_rw_direct = false
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/home/data/fastdfs
subdir_count_per_path=256
tracker_server=192.168.123.110:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
key_namespace=FastDFS
keep_alive=0
http.disabled=false
httphttp.domain_name=
http.server_port=8888
http.trunk_size=256KB
http.need_find_content_type=true
[root@db2 ~]# grep -v '^#' /etc/fdfs/client.conf |grep -v '^$'
connect_timeout=30
network_timeout=60
base_path=/home/data/fastdfs
tracker_server=192.168.123.110:22122
log_level=info
http.tracker_server_port=8080
[root@db2 ~]# grep -v '^#' /etc/fdfs/http.conf |grep -v '^$'
http.default_content_type = application/octet-stream
http.mime_types_filename=mime.types
http.anti_steal.check_token=false
http.anti_steal.token_ttl=900
http.anti_steal.secret_key=FastDFS1234567890
http.anti_steal.token_check_fail=/home/data/fastdfs/conf/anti-steal.jpg
五：启动storage，需要注意storage.conf文件最后一行为#include httpd.conf
[root@db2 ~]# mkdir -p /home/data/fastdfs
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/include/' >> /etc/ld.so.conf
[root@db2 ~]# echo '/usr/local/libevent-2.0.14/lib/' >> /etc/ld.so.conf
[root@db2 ~]# ldconfig
[root@db2 ~]# tail -2 /etc/fdfs/storage.conf
#use "#include" directive to include HTTP other settings
#include http.conf
[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_storaged /etc/fdfs/storage.conf
mkdir data path: 00 ...
mkdir data path: 01 ...
mkdir data path: 02 ...
———输出省略————
data path: /home/data/fastdfs/data, mkdir sub dir done.
[root@db2 ~]# ps -ef |grep fdfs
root 14451 1 0 16:15 ? 00:00:00 /usr/local/FastFDS/bin/fdfs_storaged
/etc/fdfs/storage.conf
root 14468 8238 0 16:16 pts/1 00:00:00 grep fdfs
[root@db2 ~]# netstat -ntpl |grep fdfs
tcp 0 0 192.168.123.20:8888 0.0.0.0:* LISTEN
14451/fdfs_storaged
tcp 0 0 192.168.123.20:23000 0.0.0.0:* LISTEN
14451/fdfs_storaged
五：测试文件上传
|
[root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload /etc/passwd [2012-07-30 16:25:59] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group: [root@db2 ~]# /usr/local/FastFDS/bin/fdfs_test /etc/fdfs/client.conf upload 2.jpg [2012-07-30 16:29:22] INFO - base_path=/home/data/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0tracker_query_storage_store_list_without_group: |
转载于:https://blog.51cto.com/xiaodong88/1329507
FastDFS分布式文件系统的安装及配置相关推荐
- Docker安装FastDFS分布式文件系统
Docker安装FastDFS分布式文件系统: 1.首先下载FastDFS文件系统的docker镜像 2.拉取镜像 3.查看镜像 4.使用docker镜像构建tracker容器(跟踪服务器,起到调度的 ...
- FastDFS分布式文件系统(详细入门级介绍)
分布式文件系统 (Distributed File System) 是一个软件/软件服务器: 这个软件可以用来管理文件: 但这个软件所管理的文件通常不是在一个服务器节点上,而是在多个服务器节点上,这些 ...
- 搭建自己的云存储空间|FastDFS分布式文件系统考虑一下？
一. 前言 最近有很多小伙伴问壹哥,大规模的文件存储该怎么做? 这个感觉很难实现呢.其实这个需求,并没有大家想的那么难以实现.今天壹哥就带着各位,用10分钟的时间来搞定这个需求.不信?那你就跟着壹哥一 ...
- 大量文件不知道 怎么管理，快试试FastDFS(分布式文件系统)
FastDFS(分布式文件系统)入门到精通 GitHub:个人GitHub Blog: 个人博客 1 什么是FastDFS 1.1 简介 FastDFS是用c语言编写的一款开源的分布式文件系统,它是由 ...
- FastDFS 分布式文件系统详解
FastDFS 分布式文件系统详解 什么是文件系统 文件系统是操作系统用于在磁盘或分区上组织文件的方法和数据结构.磁盘空间是什么样的我们并不清楚,但文件系统可以给我们呈现一个非常清晰的表象,我们可以创 ...
- FastDFS分布式文件系统使用手册
FastDFS分布式文件系统使用手册 1.概述 1.1.FastDFS简介 FastDFS是由国人余庆所开发,其项目地址:https://github.com/happyfish100,FastDFS ...
- FastDFS分布式文件系统详解
FastDFS 简介 FastDFS 是基于 C 语言开发的,是一个轻量级开源的高性能分布式文件系统.主要功能有:文件存储.文件同步.文件访问(文件上传/下载),解决了大容量的文件存储和高并发访问的问 ...
- Fastdfs分布式文件系统的应用
2019独角兽企业重金招聘Python工程师标准>>> 26 MARCH 2016 on fastdfs, 分布式文件系统 我们在实际项目中常常遇到这样的应用场景,用户需要上传图片, ...
- suse 启动oracle11g,SuSe10下Oracle11g文件系统模式安装及配置、网络配置与连接
SuSe10下Oracle11g文件系统模式安装及配置.网络配置与连接 概述 本课程主要讲解oracle数据库软件的安装及配置,以及数据库的创建过程和网络配置与连接等:同时讲解一些数据库安装过程中的常 ...
最新文章
- 手动初始化设置3PAR存储系统
- vue 数据（data）赋值问题
- lightroom安卓_安卓可以用的一款PS
- QT的QLatin1String类的使用
- python支付程序源码_支付宝推出新活动，Python脚本能让你赚的更多！（附源码）...
- 【解读】Http协议
- 麒麟OS新版本，老国产平台机器都装不了，情何以堪？
- 极易上手搭建自己日志采集服务器分析日志（winlogbeat+Elasticsearch+Kibana)
- 阿里矢量图库使用方法
- 魔兽地图编辑器自定义地图预览图片的方法
- 如何在Github上建立自己的个人博客网站详细教程
- HTML5游子吟网页的完整代码,游子吟教学完整版古诗
- 赞！WEB设计之路！网络视觉艺术发展史概览
- 使用obi fluid进行洪水模拟，持续更新~
- Android 小米应用角标
- 规则引擎解决方案浅析
- c语言深度剖析(2)—有符号与无符号
- 冬瓜你只会煮汤吗？其实还能这样吃！
- 【计算机视觉】双目测距（二）--双目标定与矫正
- 微信小程序蓝牙writeBLECharacteristicValue 1004错误问题