Decision Transformer环境安装

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling

参考了min-decision-transformer方法：min-decision-transformer

一、安装mujoco-py

.1.创建虚拟环境

conda create -n decision_transformer python=3.8
conda activate decision_transformer

注意python版本需要>=3.7，避免后续与d4rl的冲突。

2.安装libs

sudo apt-get updatesudo apt-get install gccsudo apt-get build-dep mesasudo apt-get install llvm-devsudo apt-get install freeglut3 freeglut3-devsudo apt-get install python3-devsudo apt-get install build-essentialsudo apt install curl git libgl1-mesa-dev libgl1-mesa-glx libglew-dev \libosmesa6-dev software-properties-common net-tools unzip vim \virtualenv wget xpra xserver-xorg-dev libglfw3-dev patchelf

3.安装mujoco

wget https://roboti.us/download/mujoco200_linux.zip
wget https://roboti.us/file/mjkey.txt

mkdir /root/.mujoco### mujoco 200
unzip mujoco200_linux.zip -d /home/lab1527--2/.mujoco
cp -r /home/lab1527--2/.mujoco/mujoco200_linux /home/lab1527--2/.mujoco/mujoco200mv mjkey.txt /home/lab1527--2/.mujococp -r /home/lab1527--2/.mujocoo/mujoco200/bin/* /usr/lib/

4.环境变量

在~/.bashrc中添加

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/lab1527--2/.mujoco/mujoco200/bin

5.设置mujoco_py

pip install mujoco_py==2.0.2.8

二、安装D4RL

！！！！！！！！！！！！！！！！需要注意！！！！！！！！！！！！！！

在安装好mujoco，mujoco_py的基础上，直接安装d4rl会报错！！！！！！

1.先安装两个库文件，避免报错

pip install absl-py
pip install matplotlib

2.安装 dm_control

pip install dm_control

如果不先安装dm_control，会报错：

3. 克隆D4RL仓库：

git clone https://github.com/rail-berkeley/d4rl.git

4.找到d4rl目录下的setup.py文件，

！！！！！！！！注释mujoco_py, dm_control, mjrl。！！！！！！！！！！！

注意一定要注释掉mjrl,否则会报错。

5.安装

pip install -e .

6.安装mjrl

pip install git+https://github.com/aravindr93/mjrl@master#egg=mjrl

三、检查mujoco-py and D4RL

# set mujoco env path if not already set
%env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/root/.mujoco/mujoco200/binimport gym
import d4rl # Import required to register environmentsenv = gym.make('Walker2d-v3')
env.reset()
env.step(env.action_space.sample())
env.close()
print("mujoco-py check passed")env = gym.make('walker2d-medium-v2')
env.reset()
env.step(env.action_space.sample())
env.close()
print("d4rl check passed")

四、下载D4RL数据集

# set mujoco env path if not already set
%env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/root/.mujoco/mujoco200/binimport os
import gym
import numpy as npimport collections
import pickleimport d4rldatasets = []data_dir = "./data"print(data_dir)if not os.path.exists(data_dir):os.makedirs(data_dir)for env_name in ['walker2d', 'halfcheetah', 'hopper']:for dataset_type in ['medium', 'medium-expert', 'medium-replay']:name = f'{env_name}-{dataset_type}-v2'pkl_file_path = os.path.join(data_dir, name)print("processing: ", name)env = gym.make(name)dataset = env.get_dataset()N = dataset['rewards'].shape[0]data_ = collections.defaultdict(list)use_timeouts = Falseif 'timeouts' in dataset:use_timeouts = Trueepisode_step = 0paths = []for i in range(N):done_bool = bool(dataset['terminals'][i])if use_timeouts:final_timestep = dataset['timeouts'][i]else:final_timestep = (episode_step == 1000-1)for k in ['observations', 'next_observations', 'actions', 'rewards', 'terminals']:data_[k].append(dataset[k][i])if done_bool or final_timestep:episode_step = 0episode_data = {}for k in data_:episode_data[k] = np.array(data_[k])paths.append(episode_data)data_ = collections.defaultdict(list)episode_step += 1returns = np.array([np.sum(p['rewards']) for p in paths])num_samples = np.sum([p['rewards'].shape[0] for p in paths])print(f'Number of samples collected: {num_samples}')print(f'Trajectory returns: mean = {np.mean(returns)}, std = {np.std(returns)}, max = {np.max(returns)}, min = {np.min(returns)}')with open(f'{pkl_file_path}.pkl', 'wb') as f:pickle.dump(paths, f)

Decision Transformer环境安装相关推荐

NeurIPS论文解读｜Decision Transformer: 通过序列建模解决离线强化学习问题
今天为大家推荐一篇2021年被NeurIPS收录的一篇论文. <Decision Transformer: reinforcement learning via sequence modelin ...
【强化学习论文】Decision Transformer：通过序列建模进行强化学习
Article 文献题目:Decision Transformer: Reinforcement Learning via Sequence Modeling 文献时间:2021 摘要我们引入了一个 ...
Decision Transformer 前沿追踪——万物皆可归于序列预测
引言如果想要将强化学习技术应用在某个决策领域,最重要的就是将原始问题转换为一个合理的 MDP (马尔科夫决策过程)问题,而一旦问题环境本身有一些不那么友好的"特性"(比如部分可观 ...
【论文阅读】Decision Transformer: Reinforcement Learning via Sequence Modeling
[论文阅读]Decision Transformer: Reinforcement Learning via Sequence Modeling 1 本文解决了什么问题? 本文将强化学习抽象为一个序列 ...
Online Decision Transformer
摘要最近的工作表明,离线强化学习 (RL) 可以表述为序列建模问题 (Chen et al., 2021; Janner et al., 2021),并通过类似于大规模语言建模的方法来解决. 然而, ...
2021年大数据常用语言Scala（二）：Scala开发环境安装
目录开发环境安装安装JDK 安装scala SDK 步骤具体操作安装IDEA scala插件步骤开发环境安装学习如何编写scala代码之前,需要先安装scala编译器以及开发工具 sca ...
linux学 java_[操作系统]Linux学习第二步(Java环境安装)
[操作系统]Linux学习第二步(Java环境安装) 0 2017-05-19 00:02:21 jdk版本:jdk-8u131-linux-x64.rpm 注:以下操作在root用户或具有root权 ...
ZooKeeper集群环境安装与配置
原文:出自本人的Linux博客http://blog.csdn.net/unix21/ ZooKeeper版本:3.4.5 约定:3台虚拟机前提:需要安装JDK,关于Linux环境JDK安装配置参考 ...
docker 离线安装 mysql_Oracle数据库之docker 离线环境安装oracle
本文主要向大家介绍了Oracle数据库之docker 离线环境安装oracle,通过具体的内容向大家展现,希望对大家学习Oracle数据库有所帮助. 因测试需要,需在内网的测试环境搭建一套docker ...

Decision Transformer环境安装

Decision Transformer环境安装相关推荐

最新文章

热门文章