當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

ceph对象存储折腾记

發布時間：2025/5/22 编程问答 30 豆豆

生活随笔收集整理的這篇文章主要介紹了 ceph对象存储折腾记小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

2019獨角獸企業重金招聘Python工程師標準>>>

前言

一直想弄對象存儲，以前弄過一次，不是很理解region是個什么東西，后來時間和工作上的原因沒有再折騰，這兩天閑了下來，再次折騰了一次。我是參考的ceph的中文翻譯文擋進行的部署和測試。傳送門,文檔里面介紹的和ceph本身的版本存在脫節的現象，可能初次接觸的人會因為服務啟動的問題摸不著頭腦。 本文檔只適用于ceph jewel版本的部署

關于部署

安裝ceph必要的軟件包，配置好公共密鑰和ceph mon的配置，這里我不再談了。對象存儲額外需要安裝的包是：ceph-radosgw和ceph-common 安裝完畢你的系統上應該至少存在三個命令：rados 、 radosgw 、 radosgw-admin 其中整個對象網關服務就是由radosgw來啟動的，radosgw-admin負責管理對象資源（用戶，權限，bucket），rados基本算一個比較簡單的s3客戶端（？我這里可能理解不是很精確）

配置

ceph.conf

[global] fsid = xxxxxxxxxxxxxxxxxxxxxxxxxxxx mon_initial_members = t41,t42,t45 mon_host = 192.168.168.41,192.168.168.42,192.168.168.45 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx# t56是服務器的hostname,由hostname -s命令可獲取 [client.radosgw.t56] host = your_ceph_rados_host keyring = /etc/ceph/ceph.client.radosgw.keyring rgw_socket_path = "/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock" log_file = "/data/logs/client.radosgw.gateway.log" rgw_frontends = civetweb port=80 rgw_print_continue = true

所有配置選項參考傳送門

密鑰環

#創建方式 ceph auth create client.radosgw.t56 osd 'allow rwx' mon 'allow rwx' -o /etc/ceph/ceph.client.radosgw.keyring #密鑰環的樣子 [client.radosgw.t56]key = xxxxxxxxxxxxxxxxxxxxxxxx==

服務的啟動方式

#有三種啟動方式 1. /etc/init.d/ceph-radosgw [start|stop|status|reload] 2. systemctl start ceph-radosgw 3. radosgw -c /etc/ceph/ceph.conf -n client.radosgw.t56

本質都是第三種啟動，無非寫了個腳本而已。radosgw -h可以看看其他的參數，其中-f前臺執行和--debug_ms設置調試等級有利于調試。

授權用戶

[root@t56 /data]# radosgw-admin user create --uid="xueyi28" --display-name="Xueyi" {"user_id": "xueyi28","display_name": "Xueyi","email": "","suspended": 0,"max_buckets": 1000,"auid": 0,"subusers": [],"keys": [{"user": "xueyi28","access_key": "xxxxxxxxxxxxxxx","secret_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}],"swift_keys": [],"caps": [],"op_mask": "read, write, delete","default_placement": "","placement_tags": [],"bucket_quota": {"enabled": false,"max_size_kb": -1,"max_objects": -1},"user_quota": {"enabled": false,"max_size_kb": -1,"max_objects": -1},"temp_url_keys": [] }

這里有一個要注意的，這數據格式是json格式，access_key和secret_key上可能存在反斜杠之類的轉意字符，用key的時候，注意把轉意字符處理一下，省的糾結生成的key老是驗證不過。

關于region

對象存儲設計考慮到數據中心區域的問題，這里的region就是區域的標識。比如中國西部數據中心，中國南方數據中心，不同的區域數據中心的bucket還可以異地同步（？下一步需要研究的），配合用戶的DNS，可以讓不同區域的用戶連接不同區域的數據中心數據。

[root@t56 /data]# radosgw-admin region get {"name": "default","api_name": "","is_master": "true","endpoints": [],"hostnames": [],"master_zone": "","zones": [{"name": "default","endpoints": [],"log_meta": "false","log_data": "false","bucket_index_max_shards": 0}],"placement_targets": [{"name": "default-placement","tags": []}],"default_placement": "default-placement" }[root@t56 /data]# radosgw-admin regions list {"default_info": {"default_region": "default"},"regions": ["default"] }

我這個是測試用的，就只有一個默認的default region。

數據的讀寫和bucket的使用

我這里主要說php sdk使用s3接口。ceph給的文檔里面的大多數sdk版本都是上個世紀的，亞馬遜的s3 php sdk變的亂七八糟，入門比較慢，搞半天也搞不明白。我留了一個老版本的php sdk，湊合能接到ceph的文檔。測試代碼

<\?php define('AWS_KEY', 'your_access_key'); define('AWS_SECRET_KEY', 'your_secret_key'); define('AWS_CANONICAL_ID', 'xueyi28'); define('AWS_CANONICAL_NAME', 'Xueyi'); $HOST = 'your_ceph_radosgw_host';// require the amazon sdk for php library require_once 'AWSSDKforPHP/sdk.class.php';// Instantiate the S3 class and point it at the desired host $Connection = new AmazonS3(array('key' => AWS_KEY,'secret' => AWS_SECRET_KEY,'canonical_id' => AWS_CANONICAL_ID,'canonical_name' => AWS_CANONICAL_NAME, )); $Connection->use_ssl = false; //禁用ssl $Connection->set_hostname($HOST); $Connection->enable_path_style(true); //采用path的模式，不然就是域名模式，bucket會成為根域名的子域名 $Connection->allow_hostname_override(false); $Connection->path_style = true; //sdk哪里有點問題，path_style配置老是不生效，可以調一下sdk的代碼 #$Connection->create_bucket('my-new-bucket', AmazonS3::REGION_US_E1); //region可以在代碼中進行定制，一個region對應一個地方域名。我們是default region 就對應REGION_US_E1，默認的就這個，看代碼就明白了 $ListResponse = $Connection->list_buckets(); $Buckets = $ListResponse->body->Buckets->Bucket; foreach ($Buckets as $Bucket) {echo $Bucket->Name . "\t" . $Bucket->CreationDate . "\n"; } $Connection->create_object('my-new-bucket', 'hello.txt', array('body' => "Hello World!", )); $Connection->set_object_acl('my-new-bucket', 'hello.txt', AmazonS3::ACL_PUBLIC);

php sdk傳送門核心邏輯代碼/services/s3.class.php

終端玩轉對象存儲

在終端下要用s3對象存儲，最好的工具無非是s3cmd,一般yum就可以直接安裝下來，不過這玩意要注意版本，老版本的和最新的文檔用法差別比較大。

首先要定制一下s3cmd的配置

[root@t56 ~]# s3cmd --configure Enter new values or accept defaults in brackets with Enter. Refer to user manual for detailed description of all options.Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables. Access Key: xxxxxxxxxxxxxxxxxxxxxxxx Secret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Default Region [US]: defaultEncryption password is used to protect your files from reading by unauthorized persons while in transfer to S3 Encryption password: Path to GPG program [/usr/bin/gpg]: When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer Use HTTPS protocol [No]: NoOn some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly HTTP Proxy server name: New settings:Access Key: xxxxxxxxxxxxxxxxxxxxxxxxxxSecret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxDefault Region: defaultEncryption password: Path to GPG program: /usr/bin/gpgUse HTTPS protocol: FalseHTTP Proxy server name: HTTP Proxy server port: 0Test access with supplied credentials? [Y/n] nSave settings? [y/N] y Configuration saved to '/root/.s3cfg'

定制完之后要再編輯一下/root/.s3cfg文件，把你自定義的Host寫進去，定義好你的Host和region、bucket的訪問方式給一個例子

[root@t56 ~]# cat .s3cfg [default] access_key = xxxxxxxxxxxxxxxxxxxxxxx access_token = add_encoding_exts = add_headers = bucket_location = default ca_certs_file = cache_file = check_ssl_certificate = True cloudfront_host = your_ceph_rgw_host default_mime_type = binary/octet-stream delay_updates = False delete_after = False delete_after_fetch = False delete_removed = False dry_run = False enable_multipart = True encoding = ANSI_X3.4-1968 encrypt = False expiry_date = expiry_days = expiry_prefix = follow_symlinks = False force = False get_continue = False gpg_command = /usr/bin/gpg gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s gpg_passphrase = guess_mime_type = True host_base = your_ceph_rgw_host host_bucket = your_ceph_rgw_host/%(bucket) human_readable_sizes = False ignore_failed_copy = False invalidate_default_index_on_cf = False invalidate_default_index_root_on_cf = True invalidate_on_cf = False list_md5 = False log_target_prefix = max_delete = -1 mime_type = multipart_chunk_size_mb = 15 preserve_attrs = True progress_meter = True proxy_host = proxy_port = 0 put_continue = False recursive = False recv_chunk = 4096 reduced_redundancy = False restore_days = 1 secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx send_chunk = 4096 server_side_encryption = False signature_v2 = False simpledb_host = sdb.your_ceph_rgw_host skip_existing = False socket_timeout = 300 urlencoding_mode = normal use_https = False use_mime_magic = True verbosity = WARNING website_endpoint = http://%(bucket)s.s3-website-%(location)your_ceph_rgw_host/ website_error = website_index = index.html

有幾個地方修改你得按照你自己的需求去整，需要什么就整什么，讓我們試試

[root@t56 ~]# s3cmd ls 2016-11-23 12:36 s3://my-new-bucket [root@t56 ~]# s3cmd ls s3://my-new-bucket 2016-11-23 12:38 12 s3://my-new-bucket/hello.txt [root@t56 ~]# s3cmd put test.log s3://my-new-bucket/test.log test.log -> s3://my-new-bucket/test.log [1 of 1]242 of 242 100% in 0s 3.48 kB/s done [root@t56 ~]# s3cmd setacl s3://my-new-bucket/test.log --acl-public s3://my-new-bucket/test.log: ACL set to Public [1 of 1] [root@t56 ~]# s3cmd del s3://my-new-bucket/hello.txt File s3://my-new-bucket/hello.txt deleted//遞歸授權的方式 s3cmd setacl s3://myexamplebucket.calvium.com/ --acl-public --recursive

棒極了

轉載于:https://my.oschina.net/xueyi28/blog/793600

總結

以上是生活随笔為你收集整理的ceph对象存储折腾记的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

对象
ceph