1 环境搭建
1.1 安装FastDFS
测试环境：172.16.100.133 usr:root pwd:111111
Tracker和storage安装在同一台机器上。
FastDFS版本：v3.11
下载地址：http://code.google.com/p/fastdfs/downloads/list
Libevent版本: v2.0.21
下载地址：http://libevent.org/
JavaAPI版本: v1.24 下载地址（包含source code、doc、API）
下载地址：http://code.google.com/p/fastdfs/downloads/list
1.1.1安装libevent
FastDFS内部使用libevent作为http服务器，建议还是使用apache或Nginx。如果已经安装了 libevent，请确认安装路径是 /usr ， 因为 FastDFS 在编译源程序时，需要到此目录下查找一些依赖文件，否则编译 FastDFS 会出错 。如果不是，建议首先卸载 libevent ，然后安装到 /usr 下。Ubuntu10.4默认是安装了libevent，可以到软件中心卸载掉然后按照下面介绍的方式安装。
#Tar xvf libevent-2.0.21-stable
#cd libevent-2.0.21-stable
#./configure –prefix=/usr
#make
#make install
安装track_sever
#tar zxvf FastDFS_v3.02.tar.gz
开启http支持，需要先修改make.sh文件，修改"#WITH_HTTPD=1"为"WITH_HTTPD=1"
#./make.sh
#./make.sh install
配置文件分别存放在客户端配置文件 Client.conf,文件存储服务器配置文件Strage.conf,调度服务器配置文件Tracker.conf。其中client.conf在使用Java客户端时需要在Java程序中解析获得相关信息。
1.1.2 配置track_sever
修改tracker.conf文件，修改log、group等文件存放的目录
bast_path=/home/yuqing/fastdfs ->bast_path=/usr/FastDFS
根据需要修改http端口
http.server_port=8080
开启http支持
##include http.conf -> #includehttp.conf
启动tracker_server
/usr/local/bin/fdfs_trackred/home/yuqing/FastDFS/conf/tracker.conf
查看/usr/FastDFS/logs/tracker.log，查看是否启动成功
1.1.3 安装storage_server
配置storage
修改storage.conf配置文件,定义log、group等存储目录
base_path=/home/yuqing/fastdfs ->bast_path=/usr/FastDFS
修改文件存位置，一个storage可以指定多个目录
store_path0=/home/yuqing/fastdfs-> store_path0=/usr /FastDFS
定义组（以组名区别）
group_name=group1
修改tracker_server的地址和端口，需要将所有tracker_server加入
tracker_server 192.168.209.121:22122 -> tracker_server192.168.8.235:22122 tracker_server 192.168.8.236:22122
开启http支持
##includehttp.conf –> #include http.conf
启动storage服务器
#/usr/local/bin/fdfs_storaged/home/yuqing/FastDFS/conf/storage.conf
执行时会在data文件夹下生产256个文件夹，
打开storage.log日志，查看是否执行成功
4.1.4 配置客户端配置文件
修改/usr /FastDFS/conf/client文件
base_path=/home/yuqing/fastdfs-> bast_path=/usr/FastDFS
修改tracker_server，建议添加所有track_server
开启http支持
##includehttp.conf –> #include http.conf
执行上传文件命令#/usr/local/bin/fdfs_test /usr/FastDFS/conf/client.conf upload test.txt
根据返回信息，查找/data下相应文件夹，可以看到文件上传成功。
1.2 FastDFS与Apache整合
本节主要介绍如何使用fastdfs-apache-module模块整合fastdfs和apahce；整合完毕后，客户端访问apache，apache根据配置，使用fastdfs连接tracker查询相应组内可用的stroage服务器，最后完成http请求的重定向！因而建议在每台storage服务器上都配置fastdfs和apahce的整合，各个组间配置相应的虚拟主机，还可以在前端使用nginx做负载均衡来分配用户的请求。在开始之前，需要在storage服务器上安装好apache，本例中使用编译方式安装！
1.2.1下载fastdfs-apache-module源码
[root@www ~]# wget http://fastdfs.googlecode.com/files/fastdfs-apache-module_v1.08.tar.gz
[root@www ~]# tar -zxvf fastdfs-apache-module_v1.08.tar.gz
[root@www ~]# cd fastdfs-apache-module/src/
1.2.2查看apache的编译路径和参数
[root@www src]# /home/web/apache/bin/apachectl -V
Server version: Apache/2.2.17 (Unix)
Server built: Apr 1 2011 06:11:57
Server's Module Magic Number: 20051115:25
Server loaded: APR 1.4.2, APR-Util 1.3.10
Compiled using: APR 1.4.2, APR-Util 1.3.10
Architecture: 32-bit
Server MPM: Prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/home/web/apache"
-D SUEXEC_BIN="/home/web/apache/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
1.2.3 修改makefile文件
根据apache编译情况，修改makefile文件如下；在这里不要修改INCLUDES和LIBS参数，否则编译会报错，后面使用软连接实现！
[root@www src]# grep -v '^#' Makefile |grep -v '^$'
APACHE_BASE_PATH=/home/web/apache
builddir=.
top_srcdir=$(APACHE_BASE_PATH)
top_builddir=$(APACHE_BASE_PATH)
include $(APACHE_BASE_PATH)/build/special.mk
APXS=$(APACHE_BASE_PATH)/bin/apxs
APACHECTL=$(APACHE_BASE_PATH)/bin/apachectl
CFLAGS=-Wall
DEFS=-D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='"/etc/fdfs/mod_fastdfs.conf"'
INCLUDES=-I/usr/local/include/fastdfs -I/usr/local/include/fastcommon/
LIBS=-L/usr/local/lib -lfastcommon -lfdfsclient
SH_LIBS=-lfastcommon -lfdfsclient
all: local-shared-build
install: install-modules-yes
if [ ! -f /etc/fdfs/mod_fastdfs.conf ]; then cp mod_fastdfs.conf /etc/fdfs/; fi
clean:
-rm -f mod_fastdfs.o mod_fastdfs.lo mod_fastdfs.slo mod_fastdfs.la
test: reload
lynx -mime_header http://localhost/fastdfs
reload: install restart
start:
$(APACHECTL) start
restart:
$(APACHECTL) restart
stop:
$(APACHECTL) stop
1.2.4 对fastfds目录下的lib和include目录做软连接
[root@www src]# ln -s /usr/local/FastFDS/lib/libf* /usr/local/lib
[root@www src]# ln -s /usr/local/FastFDS/include/fastdfs /usr/local/include/
[root@www src]# ln -s /usr/local/FastFDS/include/fastcommon/ /usr/local/include/
1.2.5编译安装
[root@www src]# make
/home/web/apache/build/libtool --silent --mode=link gcc -g -O2 -pthread -Wall -lfastcommon -lfdfsclient -o
mod_fastdfs.la -rpath /home/web/apache/modules -module -avoid-version mod_fastdfs.lo
[root@www src]# make install
/home/web/apache/build/libtool --silent --mode=install cp mod_fastdfs.la /home/web/apache/modules/
make[1]: Entering directory `/root/fastdfs-apache-module/src'
/home/web/apache/build/libtool --silent --mode=install cp mod_fastdfs.la /home/web/apache/modules/
make[1]: Leaving directory `/root/fastdfs-apache-module/src'
if [ ! -f /etc/fdfs/mod_fastdfs.conf ]; then cp mod_fastdfs.conf /etc/fdfs/; fi
[root@www src]# ls /home/web/apache/modules/*fast*
/home/web/apache/modules/mod_fastcgi.so
/home/web/apache/modules/mod_fastdfs.so
1.2.6修改apache配置文件
让虚拟主机下的/M00所有请求都交给fastdfs模块处理
[root@www src]# tail /home/web/apache/conf/httpd-vhosts.conf
#test fastdfs
LoadModule fastdfs_module modules/mod_fastdfs.so
<VirtualHost *:80>
Servername img.yang.com
<Location /M00>
sethandler fastdfs
</Location>
</VirtualHost>
1.2.7修改mod_fastdfs.conf文件
指定tracker服务器的ip和端口
[root@www src]# grep -v '^#' /etc/fdfs/mod_fastdfs.conf |grep -v '^$'
connect_timeout=2
network_timeout=30
base_path=/home/data/fastdfs
tracker_server=192.168.123.110:22122
storage_server_port=23000
group_name=group1
url_have_group_name = false
store_path_count=1
store_path0=/home/data/fastdfs
log_level=debug
log_filename=
response_mode=redirect
if_alias_prefix=
http.need_find_content_type=false
备注：
当apache服务器不为storage节点时，apache接受到的用户请求将会跳转！
[root@www fdfs]# tail -f /home/web/apache/logs/access_log
192.168.123.102 - - [01/Aug/2012:11:46:08 +0800] "GET/M00/00/00/wKh7FFAYiID1TnRyAAAA4VuHcSQ5892200 HTTP/1.1" 302 -
这个时候，若组内其他可用的storage服务器没有配置fastdfs同apache或者nginx的模块整合，则用户无法访问相应的资源，因而建议所有的storage服务器都进行整合，关于fastdfs同nginx的整合，可参考官方文档或自行google！
[root@www fdfs]# tail -f /home/web/apache/logs/error_log
[2012-08-01 11:46:08] DEBUG - file: common.c, line: 622, redirect tohttp://192.168.123.20/M00/00/00/wKh7FFAYiID1TnRyAAAA4VuHcSQ5892200?redirect=1
2 Java客户端上传下载文件实例
2.1.1上传文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
|
package
com.jncumter.fastdfs;
import
java.io.File;
import
java.io.FileInputStream;
import
org.csource.common.NameValuePair;
import
org.csource.fastdfs.ClientGlobal;
import
org.csource.fastdfs.ServerInfo;
import
org.csource.fastdfs.StorageClient;
import
org.csource.fastdfs.StorageServer;
import
org.csource.fastdfs.TrackerClient;
import
org.csource.fastdfs.TrackerServer;
/***
*
* @author jncumter
* @date 2013-09-18 16:31
* Test Upload of FastDFS
*/
publicclassTestUpload {
publicstaticvoid main(String[] args)
throws
Exception {
//getting the config file of Java Client and initthe client global
StringclassPath = newFile(TestUpload.
class
.getResource(
"/"
).getFile()).getCanonicalPath();
StringconfigFilePath = classPath + File.separator +
"client.conf"
;
System.out.println(
"conf file of Client"
+ configFilePath);
ClientGlobal.init(configFilePath);
//getting connection of FastDFS
TrackerClienttrackerClient = newTrackerClient();
TrackerServertrackerServer = trackerClient.getConnection();
StorageServerstorageServer =
null
;
StorageClientstorageClient = newStorageClient(trackerServer,storageServer);
//meta data setting
NameValuePair[]meta_list = newNameValuePair[
3
];
meta_list[
0
]= newNameValuePair(
"width"
,
"120"
);
meta_list[
1
]= newNameValuePair(
"heigth"
,
"120"
);
meta_list[
2
]= newNameValuePair(
"author"
,
"gary"
);
Filefile = newFile(
"F:\\1.jpg"
);
FileInputStreamfis = newFileInputStream(file);
byte
[] file_buff =
null
;
if
(fis !=
null
) {
int
len = fis.available();
file_buff= newbyte[len];
fis.read(file_buff);
}
System.out.println(
"file length: "
+ file_buff.length);
Stringgroup_name =
null
;
//get store storages
StorageServer[]storageServers = trackerClient.getStoreStorages(trackerServer, group_name);
if
(storageServers ==
null
) {
System.err.println(
"get store storage servers fail, error code: "
+storageClient.getErrorCode());
}
else
{
System.err.println(
"store storage servers count: "
+storageServers.length);
for
(
int
k =
0
; k <storageServers.length;k++) {
System.err.println(k
+
1
+
". "
+storageServers[k].getInetSocketAddress().getAddress()
.getHostAddress()+
":"
+storageServers[k].getInetSocketAddress().getPort());
}
System.err.println(
""
);
}
long
startTime = System.currentTimeMillis();
// upload files
String[]results = storageClient.upload_file(file_buff,
"jpg"
,
meta_list);
System.out.println(
"upload_file time used: "
+(System.currentTimeMillis() - startTime) +
" ms"
);
if
(results ==
null
) {
System.err.println(
"upload file fail, error code: "
+storageClient.getErrorCode());
return
;
}
//get groupName and remoteFileName
group_name= results[
0
];
Stringremote_filename = results[
1
];
System.err.println(
"group_name: "
+ group_name +
", remote_filename: "
+remote_filename);
System.err.println(storageClient.get_file_info(group_name,
remote_filename));
//get storage sever of this file
ServerInfo[]servers = trackerClient.getFetchStorages(trackerServer,
group_name,remote_filename);
if
(servers ==
null
) {
System.err.println(
"get storage servers fail, error code: "
+trackerClient.getErrorCode());
}
else
{
System.err.println(
"storage servers count: "
+ servers.length);
for
(
int
k =
0
; k < servers.length; k++) {
System.err.println(k +
1
+
". "
+servers[k].getIpAddr() +
":"
+servers[k].getPort());
}
System.err.println(
""
);
}
}
}
|
输出：
conf file ofClientF:\workspace\px\TestFastDFS\bin\client.conf
file length: 35273
store storage servers count: 1
1. 172.16.100.133:23000
upload_file time used: 54 ms
group_name: group1, remote_filename:M00/00/00/rBBkhVI1pfiAWkXHAACJyRZF0Xk289.jpg
source_ip_addr = 172.16.100.133,file_size = 35273, create_timestamp = 2013-09-15 20:20:08, crc32 = 373674361
storage servers count: 1
1.172.16.100.133:23000
查看服务器相应存储目录：
[root@phicomm-sd-3 00]# cd/usr/FastDFS/data/00/00
[root@phicomm-sd-3 00]# ls
rBBkhVI1P8SAe3EiAACJyRZF0Xk717.jpg rBBkhVIm51qAVv6FAAAGkCLo-iI8997_big.sh rBBkhVIm6Q2ACm6oABf5MYLXTow198_big.JPG rBBkhVIm7q-AE9-bABf5MYLXTow905_big.JPG rBBkhVIv30GAEaZ4AACJyRZF0Xk263.jpg
rBBkhVI1P8SAe3EiAACJyRZF0Xk717.jpg-m rBBkhVIm51qAVv6FAAAGkCLo-iI8997_big.sh-m rBBkhVIm6Q2ACm6oABf5MYLXTow198_big.JPG-m rBBkhVIm7q-AE9-bABf5MYLXTow905_big.JPG-m rBBkhVIv30GAEaZ4AACJyRZF0Xk263.jpg-m
rBBkhVI1pfiAWkXHAACJyRZF0Xk289.jpg rBBkhVIm51qAVv6FAAAGkCLo-iI8997.sh rBBkhVIm6Q2ACm6oABf5MYLXTow198.JPG rBBkhVIm7q-AE9-bABf5MYLXTow905.JPG rBBkhVIv5tKAOI4NAACJyRZF0Xk206.jpg
rBBkhVI1pfiAWkXHAACJyRZF0Xk289.jpg-m rBBkhVIm51qAVv6FAAAGkCLo-iI8997.sh-m rBBkhVIm6Q2ACm6oABf5MYLXTow198.JPG-m rBBkhVIm7q-AE9-bABf5MYLXTow905.JPG-m rBBkhVIv5tKAOI4NAACJyRZF0Xk206.jpg-m
2.1.2下载文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
|
package
com.jncumter.fastdfs;
import
java.io.File;
import
java.io.FileOutputStream;
import
org.csource.fastdfs.ClientGlobal;
import
org.csource.fastdfs.FileInfo;
import
org.csource.fastdfs.StorageClient;
import
org.csource.fastdfs.StorageServer;
import
org.csource.fastdfs.TrackerClient;
import
org.csource.fastdfs.TrackerServer;
/**
*
* @author jncumter
* @date 2013-09-18 16:31
* Test Download of FastDFS
*/
publicclassTestDownload {
publicstaticvoid main(String[] args)
throws
Exception {
StringclassPath = newFile(TestDownload.
class
.getResource(
"/"
).getFile()).getCanonicalPath();
StringconfigFilePath = classPath + File.separator +
"client.conf"
;
ClientGlobal.init(configFilePath);
TrackerClienttrackerClient = newTrackerClient();
TrackerServertrackerServer = trackerClient.getConnection();
StorageServerstorageServer =
null
;
StorageClientstorageClient = newStorageClient(trackerServer,storageServer);
//groupName and remoteFileName should exist
Stringgroup_name =
"group1"
;
Stringremote_filename =
"M00/00/00/rBBkhVIv5tKAOI4NAACJyRZF0Xk206.jpg"
;
FileInfofi = storageClient.get_file_info(group_name, remote_filename);
File file =
new
File(
"F:\\2.jpg"
);
FileOutputStream fos = newFileOutputStream(file);
byte
[] file_buff =storageClient.download_file(group_name, remote_filename);
if
(file_buff !=
null
){
fos.write(file_buff);
}
fos.close();
StringsourceIpAddr = fi.getSourceIpAddr();
long
size =fi.getFileSize();
System.out.println(
"ip:"
+ sourceIpAddr +
",size:"
+ size);
}
}
|
输出：
ip:172.16.100.133,size:35273
查看本机相应目录，可以看到文件下载成功。
本文转自 gaochaojs 51CTO博客，原文链接：http://blog.51cto.com/jncumter/1308558，如需转载请自行联系原作者
