Rails下cloud datastore的使用

背景

部门有一个项目要用Ruby做 WebAPI，DB使用关系型数据库Cloud Sql和非关系型数据库Cloud Datastore 。

还不了解Ruby On Rails和CloudDatastore的请参考下面的链接。

http://guides.ruby-china.org/

https://thinkit.co.jp/story/2015/02/05/5594

1、 Windows下开发环境构建

1.1、构建Ruby On Rails 开发环境

① Ruby安装

http://www.runoob.com/ruby/ruby-installation-windows.html

② Rail安装

从CMD提示窗口输入指令：gem install rails 开始安装rails。

注意：在中国直接安装会提示错误，因为gem默认的安装源https://rubygems.org在国内不能访问，还好先辈们早已为我们新搭建了一个gem安装源http://gems.ruby-china.org。下面是切换安装源的具体步骤，gem安装源切换之后，Rails安装就可以顺利进行了。

第一步：删除默认安装源

CMD提示窗中输入”gem sources -remove https://rubygems.org/”，回车换行。

第二步：添加新的安装源

CMD提示窗输入“gem sources -add  http://gems.ruby-china.org/”，回车换行。

③ ruby的IDE Ruby Mine安装

http://qiita.com/HALU5071/items/6d6a39e44865d8d04de8

1.2、搭建开发用Cloud Datastore 数据库

① 安装Google Cloud SDK

http://www.apps-gcp.com/google-cloud-sdk-install

② 安装cloud-datastore-emulator

CMD提示窗中输入“gcloud components install cloud-datastore-emulator”，回车换行。

③ 创建一个开发用的DB:dev_datastore

CMD提示窗中输入“cloud_datastore_emulator create db/dev_datastore”,回车换行。

前提条件：CMD当前运行目录下要先创建一个db的文件夹。

创建好的DB文件结构图如下：

2、 WEB API工程创建

在CMD提示窗中输入“rails new my-first-api --api”，回车换行。

创建好的API工程结构见下图，rails执行结果见附录1。

フォルダ構造
		説明
Gemfile		gemの依存関係を指定できるファイル
README.rdoc		説明書
Rakefile	ターミナルから実行可能なタスク
config.ru		Rackの設定
app/		アプリケーションを格納するディレクトリ
		主要なプログラムはこの配下に格納
	controllers/	コントローラを格納するディレクトリ
	controllers/application_controller.rb	アプリケーションで共通のコントローラ
	models/	モデルを格納するディレクトリ
config/		プロジェクトの設定ファイルを格納するディレクトリ
config/environments/		環境単位の設定ファイルを格納するディレクトリ
config/initializers/		初期化ファイルを格納するディレクトリ
config/locales/		辞書ファイルを格納するディレクトリ
db/		データベースの設定ファイルを格納するディレクトリ
doc/		ドキュメントを格納するディレクトリ
lib/		複数のアプリケーション間で共有するライブラリを格納するディレクトリ
	tasks/	自分で生成したRakefileを格納するディレクトリ
log/		ログファイルが格納されるディレクトリ。ログファイルはアプリケーションと環境ごとに作成される
public/		Web上に公開するファイルを格納するディレクトリ
tmp/		キャッシュなど、一時的なファイルを格納されるディレクトリ
test/		アプリケーションのテストに使うファイルを格納するディレクトリ
vendor/		ライブラリや他のアプリケーションで共有するような外部ライブラリを格納するディレクトリ

在附录1的最后我们看到，bundle install 并没有成功。原因是项目的gem安装源出了问题。打开Gemfile把第一行的source 'https://rubygems.org'替换成source 'http://gems.ruby-china.org'。然后在CMD里重新执行bundle install命令，这样项目需要的gem就能成功安装。

3、 新建cloud Datastore 的dataset对象

① Gem安装

在Gemfile 中追加

gem 'google-cloud-datastore'

cmd里重新执行bundle install命令。

② 配置文件

在config/initializers/目录下新建cloud_datastore.rb文件，文件内容如下图所示。

从图中我们可以看到cloud datastore 配置了三种环境下创建daset的参数，这些参数我们都放在了config/database.yml里。下面是database.yml里开发环境的配置信息，datastore 安装的主机和datastore的数据库名，这样我们就可以使用datastore了。

development:        host: 'localhost:8180'        project: ' dev_datastore '

cloud_datastore.rb

4、 非关系型数据库 中ActiveModel 的使用

如果我们想要cloud datastore和关系型数据库一样，可以方便快捷使用model（ActiveModel），怎么办？下面就是你想要的答案。

① 追加文件

在config/initializers/目录下新建active_model_cloud_datastore.rb文件。

文件内容参照附录2。

② model Class写法

require_relative "../../config/initializers/active_model_cloud_datastore" 

class Customerinclude ActiveModelCloudDatastoreattr_accessor :customer_code, :customer_name

def attributes%w(code customer_name)# 注意多个字段之间是半角空格来区分的end

end

这样我们就可以使用activeModel很多功能，

#检索所有数据

@customers = Customer.all

#保存一条新的数据

@customer = Customer.new

@customer.save

这里就不做详细说明，具体参照附录2里的各种方法。

5、 spec测试

1. 1.   
2. 2.   
3. 3.   
4. 4.   
5. 5.   

① 测试环境安装

在Gemfile中追加下面的gem。

group :development, :test do # Testgem 'rspec-rails', '~> 3.0'gem 'rails-controller-testing'end

然后在cmd工程目录my-first-api下执行bundle install，安装追加的gem。

② 测试环境初期化

在cmd工程目录my-first-api下执行

bundle exec rails generate rspec:install

会生成下面的文件

create  .rspec

create  spec

create  spec/spec_helper.rb

create  spec/rails_helper.rb

③ controller测试

参考下面链接

http://qiita.com/shizuma/items/84e07e558abd6593df15

http://blog.naichilab.com/entry/2016/01/19/011514

④ model测试

参考下面的测试思路。

http://qiita.com/shizuma/items/c7b8d7b91e8f325f8ad9

⑤ Mock使用

参考下面链接

http://qiita.com/jnchito/items/640f17e124ab263a54dd

6、 测试覆盖率报告

①  环境搭建

在Gemfile中追加下面的gem。

group :development, :test dogem 'simplecov'end

然后在cmd工程目录my-first-api下执行bundle install，安装追加的gem。

② spec_helper.rb修改

在spec_helper.rb头部插入下面的语句，覆盖率测试中要过滤掉spec下的测试代码。

require 'simplecov'SimpleCov.start doadd_filter "/spec/ "end

③ 查看报告

在cmd工程目录my-first-api下，再次执行rspec spec命令，覆盖率报告就会自动生成再demo/coverage下，用google chrome浏览器打开index.html，就可以看到详细的信息。下面是一个覆盖率报告的截图。

7、 结束语

上面我们讲述的是Ruby下怎么使用cloud datastore的开发和测试，在Google Cloud Platform上怎么部署产品还有待下一步探索。期间遇到的各种技术问题难题，为了解决这些问题，调查的网站以日语和英语为主，总结的时候也使用了很多日语网站，由于时间有限，没能一一翻译过来，给不懂日语的朋友带来不少困难表示歉意。

附录1

API工程创建文件list

create

create  README.md

create  Rakefile

create  config.ru

create  .gitignore

create  Gemfile

create  app

create  app/assets/config/manifest.js

create  app/assets/javascripts/application.js

create  app/assets/javascripts/cable.js

create  app/assets/stylesheets/application.css

create  app/channels/application_cable/channel.rb

create  app/channels/application_cable/connection.rb

create  app/controllers/application_controller.rb

create  app/helpers/application_helper.rb

create  app/jobs/application_job.rb

create  app/mailers/application_mailer.rb

create  app/models/application_record.rb

create  app/views/layouts/application.html.erb

create  app/views/layouts/mailer.html.erb

create  app/views/layouts/mailer.text.erb

create  app/assets/images/.keep

create  app/assets/javascripts/channels

create  app/assets/javascripts/channels/.keep

create  app/controllers/concerns/.keep

create  app/models/concerns/.keep

create  bin

create  bin/bundle

create  bin/rails

create  bin/rake

create  bin/setup

create  bin/update

create  config

create  config/routes.rb

create  config/application.rb

create  config/environment.rb

create  config/secrets.yml

create  config/cable.yml

create  config/puma.rb

create  config/environments

create  config/environments/development.rb

create  config/environments/production.rb

create  config/environments/test.rb

create  config/initializers

create  config/initializers/application_controller_renderer.rb

create  config/initializers/assets.rb

create  config/initializers/backtrace_silencers.rb

create  config/initializers/cookies_serializer.rb

create  config/initializers/cors.rb

create  config/initializers/filter_parameter_logging.rb

create  config/initializers/inflections.rb

create  config/initializers/mime_types.rb

create  config/initializers/new_framework_defaults.rb

create  config/initializers/session_store.rb

create  config/initializers/wrap_parameters.rb

create  config/locales

create  config/locales/en.yml

create  config/boot.rb

create  config/database.yml

create  db

create  db/seeds.rb

create  lib

create  lib/tasks

create  lib/tasks/.keep

create  lib/assets

create  lib/assets/.keep

create  log

create  log/.keep

create  public

create  public/404.html

create  public/422.html

create  public/500.html

create  public/apple-touch-icon-precomposed.png

create  public/apple-touch-icon.png

create  public/favicon.ico

create  public/robots.txt

create  test/fixtures

create  test/fixtures/.keep

create  test/fixtures/files

create  test/fixtures/files/.keep

create  test/controllers

create  test/controllers/.keep

create  test/mailers

create  test/mailers/.keep

create  test/models

create  test/models/.keep

create  test/helpers

create  test/helpers/.keep

create  test/integration

create  test/integration/.keep

create  test/test_helper.rb

create  tmp

create  tmp/.keep

create  tmp/cache

create  tmp/cache/assets

create  vendor/assets/stylesheets

create  vendor/assets/stylesheets/.keep

remove  app/assets

remove  lib/assets

remove  tmp/cache/assets

remove  vendor/assets

remove  app/helpers

remove  test/helpers

remove  app/views/layouts/application.html.erb

remove  public/404.html

remove  public/422.html

remove  public/500.html

remove  public/apple-touch-icon-precomposed.png

remove  public/apple-touch-icon.png

remove  public/favicon.ico

remove  app/assets/javascripts

remove  config/initializers/assets.rb

remove  config/initializers/session_store.rb

remove  config/initializers/cookies_serializer.rb

Fetching gem metadata from https://rubygems.org/..........

Fetching version metadata from https://rubygems.org/..

Fetching dependency metadata from https://rubygems.org/.

Resolving dependencies...

Using rake 12.0.0

Using i18n 0.7.0

Using minitest 5.10.1

Using thread_safe 0.3.5

Using builder 3.2.2

Using erubis 2.7.0

Using mini_portile2 2.1.0

Using rack 2.0.1

Using nio4r 1.2.1

Using websocket-extensions 0.1.2

Using mime-types-data 3.2016.0521

Using arel 7.1.4

Using bundler 1.13.6

Using method_source 0.8.2

Using puma 3.6.2

Using thor 0.19.4

Using sqlite3 1.3.12

Gem::RemoteFetcher::FetchError: SSL_connect returned=1 errno=0 state=SSLv3 read

server certificate B: certificate verify failed

(https://rubygems.org/gems/concurrent-ruby-1.0.4.gem)

An error occurred while installing concurrent-ruby (1.0.4), and Bundler cannot

continue.

Make sure that `gem install concurrent-ruby -v '1.0.4'` succeeds before

bundling.

附录2

active_model_cloud_datastore.rb文件内容：

# frozen_string_literal: true

require_relative 'cloud_datastore'require 'active_model'require 'active_support'

# Integrates ActiveModel with the Google::Cloud::Datastoremodule ActiveModelCloudDatastoreextend ActiveSupport::Concerninclude ActiveModel::Modelinclude ActiveModel::Dirtyinclude ActiveModel::Validationsinclude ActiveModel::Validations::Callbacks

included doprivate_class_method :query_options, :query_sort, :query_property_filterdefine_model_callbacks :save, :update, :destroyattr_accessor :idend

def attributes[]end

# Used by ActiveModel for determining polymorphic routing.def persisted?id.present?end

# Resets the ActiveModel::Dirty tracked changesdef reload!clear_changes_informationend

# Updates attribute values on the ActiveModel::Model object with the provided params.# Example, such as submitted form params.## @param [Hash] paramsdef update_model_attributes(params)params.each do |name, value|send "#{name}=", value if respond_to? "#{name}="endend

# Builds the Cloud Datastore entity with attributes from the Model object.## @return [Entity] the updated Google::Cloud::Datastore::Entitydef build_entity(parent = nil)entity = CloudDatastore.dataset.entity(self.class.name, id)entity.key.parent = parent if parentattributes.each do |attr|entity[attr] = instance_variable_get("@#{attr}")endentityend

def save(parent = nil)run_callbacks :save doif valid?entity = build_entity(parent)success = self.class.retry_on_exception? { CloudDatastore.dataset.save(entity) }if successself.id = entity.key.idreturn trueendendfalseendend

def update(params)run_callbacks :update doupdate_model_attributes(params)if valid?entity = build_entityself.class.retry_on_exception? { CloudDatastore.dataset.save(entity) }elsefalseendendend

def destroyrun_callbacks :destroy dokey = CloudDatastore.dataset.key(self.class.name, id)self.class.retry_on_exception? { CloudDatastore.dataset.delete(key) }endend

# Methods defined here will be class methods whenever we 'include DatastoreUtils'.module ClassMethods# Queries all objects from Cloud Datastore by named kind and using the provided options.## @param [Hash] options the options to construct the query with.## @option options [Google::Cloud::Datastore::Key] :ancestor filter for inherited results# @option options [Hash] :where filter, Array in the format [name, operator, value]## @return [Array<Model>] an array of ActiveModel results.def all(options = {})query = CloudDatastore.dataset.query(name)query.ancestor(options[:ancestor]) if options[:ancestor]query_property_filter(query, options)entities = retry_on_exception { CloudDatastore.dataset.run(query) }from_entities(entities.flatten)end

# Queries objects from Cloud Datastore in batches by named kind and using the provided options.# When a limit option is provided queries up to the limit and returns results with a cursor.## @param [Hash] options the options to construct the query with. See build_query for options.## @return [Array<Model>, String] an array of ActiveModel results and a cursor that can be used# to query for additional results.def find_in_batches(options = {})next_cursor = nilquery = build_query(options)if options[:limit]entities = retry_on_exception { CloudDatastore.dataset.run(query) }next_cursor = entities.cursor if entities.size == options[:limit]elseentities = retry_on_exception { CloudDatastore.dataset.run(query) }endmodel_entities = from_entities(entities.flatten)return model_entities, next_cursorend

# Retrieves an entity by key and by an optional parent.## @param [Integer or String] id_or_name id or name value of the entity Key.# @param [Google::Cloud::Datastore::Key] parent the parent Key of the entity.## @return [Entity, nil] a Google::Cloud::Datastore::Entity object or nil.def find_entity(id_or_name, parent = nil)key = CloudDatastore.dataset.key(name, id_or_name)key.parent = parent if parentretry_on_exception { CloudDatastore.dataset.find(key) }end

# Find object by ID.## @return [Model, nil] an ActiveModel object or nil.def find(id)entity = find_entity(id.to_i)from_entity(entity)end

# Find object by parent and ID.## @return [Model, nil] an ActiveModel object or nil.def find_by_parent(id, parent)entity = find_entity(id.to_i, parent)from_entity(entity)end

def from_entities(entities)entities.map { |entity| from_entity(entity) }end

# Translates between Google::Cloud::Datastore::Entity objects and ActiveModel::Model objects.## @param [Entity] entity from Cloud Datastore# @return [Model] the translated ActiveModel object.def from_entity(entity)return if entity.nil?model_entity = newmodel_entity.id = entity.key.id unless entity.key.id.nil?model_entity.id = entity.key.name unless entity.key.name.nil?entity.properties.to_hash.each do |name, value|model_entity.send "#{name}=", valueendmodel_entity.reload!model_entityend

def exclude_from_index(entity, boolean)entity.properties.to_h.keys.each do |value|entity.exclude_from_indexes! value, booleanendend

# Constructs a Google::Cloud::Datastore::Query.## @param [Hash] options the options to construct the query with.## @option options [Google::Cloud::Datastore::Key] :ancestor filter for inherited results# @option options [String] :cursor sets the cursor to start the results at# @option options [Integer] :limit sets a limit to the number of results to be returned# @option options [String] :order sort the results by property name# @option options [String] :desc_order sort the results by descending property name# @option options [Array] :select retrieve only select properties from the matched entities# @option options [Hash] :where filter, Array in the format [name, operator, value]## @return [Query] a datastore query.def build_query(options = {})query = CloudDatastore.dataset.query(name)query_options(query, options)end

def retry_on_exception?retry_count = 0sleep_time = 0.5 # 0.5, 1, 2, 4 second between retriesbeginyieldrescue => eputs "\e[33m[#{e.message.inspect}]\e[0m"puts 'Rescued exception, retrying...'sleep sleep_timesleep_time *= 2retry_count += 1return false if retry_count > 3retryendtrueend

def retry_on_exceptionretry_count = 0sleep_time = 0.5 # 0.5, 1, 2, 4 second between retriesbeginyieldrescue => eputs "\e[33m[#{e.message.inspect}]\e[0m"puts 'Rescued exception, retrying...'sleep sleep_timesleep_time *= 2retry_count += 1raise e if retry_count > 3retryendend

def log_google_cloud_erroryieldrescue Google::Cloud::Error => eputs "\e[33m[#{e.message.inspect}]\e[0m"raise eend

# private

def query_options(query, options)query.ancestor(options[:ancestor]) if options[:ancestor]query.cursor(options[:cursor]) if options[:cursor]query.limit(options[:limit]) if options[:limit]query_sort(query, options)query.select(options[:select]) if options[:select]query_property_filter(query, options)end

# Adds sorting to the results by a property name if included in the options.def query_sort(query, options)query.order(options[:order]) if options[:order]query.order(options[:desc_order], :desc) if options[:desc_order]queryend

# Adds property filters to the query if included in the options.# Accepts individual or nested Arrays:# [['superseded', '=', false], ['email', '=', 'something']]def query_property_filter(query, options)if options[:where]opts = options[:where]if opts[0].is_a?(Array)opts.each do |opt|query.where(opt[0], opt[1], opt[2]) unless opt.nil?endelsequery.where(opts[0], opts[1], opts[2])endendqueryendend
end