오픈스택 (Openstack) 설치 (Minimal Pike : Controller node, Compute node, Block Storage node, Dash board )



=============
사전준비 설치
=============
* 난 openstack에 사용되는 계정의 비번은 demo01로하고
  mysql에 들어가는 계정의 비번은 openstack.org의 설치 가이드대로 했다
  controller 노드 = openstack01
  compute 노드 = openstack02
###########################controller openstack01
yum install chrony
vi /etc/chrony.conf
server "타임서버아이피" iburst
allow 10.0.0.0/24
systemctl enable chronyd.service
systemctl restart chronyd.service
chronyc sources
###########################compute openstack02, openstack03
yum install chrony
vi /etc/chrony.conf
server openstack01 iburst
systemctl enable chronyd.service
systemctl restart chronyd.service
chronyc sources
###########################controller openstack01
yum upgrade -y
yum install centos-release-openstack-pike
yum install https://rdoproject.org/repos/rdo-release.rpm
yum upgrade
yum install python-openstackclient
yum install openstack-selinux
========================
yum install mariadb mariadb-server python2-PyMySQL
vi /etc/my.cnf.d/openstack.cnf
[mysqld]
bind-address = 10.0.0.44
default-storage-engine = innodb
innodb_file_per_table = on
max_connections = 4096
collation-server = utf8_general_ci
character-set-server = utf8
systemctl enable mariadb.service
systemctl start mariadb.service
mysql_secure_installation
===================
yum install rabbitmq-server
systemctl enable rabbitmq-server.service
systemctl start rabbitmq-server.service
rabbitmqctl add_user openstack RABBIT_PASS
rabbitmqctl set_permissions openstack ".*" ".*" ".*"
=====================
yum install memcached python-memcached
vi /etc/sysconfig/memcached
 OPTIONS="-l 127.0.0.1,::1,openstack01"
systemctl enable memcached.service
systemctl start memcached.service
============================
yum install etcd
vi /etc/etcd/etcd.conf
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://10.0.0.44:2380"
ETCD_LISTEN_CLIENT_URLS="http://10.0.0.44:2379"
ETCD_NAME="controller"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.0.44:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.44:2379"
ETCD_INITIAL_CLUSTER="controller=http://10.0.0.44:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-01"
ETCD_INITIAL_CLUSTER_STATE="new"

systemctl enable etcd
systemctl start etcd
====================================
=============
글랜스 설치
=============
mysql -u root -p
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \
IDENTIFIED BY 'KEYSTONE_DBPASS';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' \
IDENTIFIED BY 'KEYSTONE_DBPASS';
flush privileges;
==========================

yum install openstack-keystone httpd mod_wsgi
vi /etc/keystone/keystone.conf
connection = mysql+pymysql://keystone:KEYSTONE_DBPASS@openstack01/keystone
provider = fernet
su -s /bin/sh -c "keystone-manage db_sync" keystone
keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
keystone-manage credential_setup --keystone-user keystone --keystone-group keystone

keystone-manage bootstrap --bootstrap-password ADMIN_PASS \
  --bootstrap-admin-url http://openstack01:35357/v3/\
  --bootstrap-internal-url http://openstack01:5000/v3/\
  --bootstrap-public-url http://openstack01:5000/v3/ \
  --bootstrap-region-id RegionOne
=================
vi /etc/httpd/conf/httpd.conf
ServerName controller
ln -s /usr/share/keystone/wsgi-keystone.conf /etc/httpd/conf.d/

=========================
systemctl enable httpd.service
systemctl start httpd.service
아래내용 /etc/profile에 add
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_AUTH_URL=http://openstack01:35357/v3
export OS_IDENTITY_API_VERSION=3
================
openstack project create --domain default \
  --description "Service Project" service

openstack project create --domain default \
  --description "Demo Project" demo
openstack user create --domain default \
  --password-prompt demo
openstack role create user
openstack role add --project demo --user demo user

==========================
=============
글랜스 설치
=============
mysql -u root -p
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' \
  IDENTIFIED BY 'GLANCE_DBPASS';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' \
  IDENTIFIED BY 'GLANCE_DBPASS';
flush privileges;

openstack user create --domain default --password-prompt glance

openstack role add --project service --user glance admin

openstack service create --name glance \
  --description "OpenStack Image" image
openstack endpoint create --region RegionOne \
  image public http://openstack01:9292
openstack endpoint create --region RegionOne \
  image internal http://openstack01:9292
openstack endpoint create --region RegionOne \
  image admin http://openstack01:9292
===================================
yum install openstack-glance
vi /etc/glance/glance-api.conf
[database]
connection = mysql+pymysql://glance:GLANCE_DBPASS@openstack01/glance
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = demo01
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = demo01
[paste_deploy]
flavor = keystone
[glance_store]
stores = file,http
default_store = file
filesystem_store_datadir = /var/lib/glance/images/
=========================

vi /etc/glance/glance-registry.conf
[database]
connection = mysql+pymysql://glance:GLANCE_DBPASS@openstack01/glance
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = demo01
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = demo01
[paste_deploy]
flavor = keystone
====================
 systemctl enable openstack-glance-api.service \
  openstack-glance-registry.service
 systemctl start openstack-glance-api.service \
  openstack-glance-registry.service
su -s /bin/sh -c "glance-manage db_sync" glance
======================
wget http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
openstack image create "cirros" \
  --file cirros-0.3.5-x86_64-disk.img \
  --disk-format qcow2 --container-format bare \
  --public

openstack image list

=====================
=============
노바 설치
=============

mysql -u root -p

CREATE DATABASE nova_api;
CREATE DATABASE nova;
CREATE DATABASE nova_cell0;
GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';
flush privileges;

openstack user create --domain default --password-prompt nova
openstack role add --project service --user nova admin
openstack service create --name nova \
  --description "OpenStack Compute" compute
openstack endpoint create --region RegionOne \
  compute public http://openstack01:8774/v2.1
openstack endpoint create --region RegionOne \
  compute internal http://openstack01:8774/v2.1
openstack endpoint create --region RegionOne \
  compute admin http://openstack01:8774/v2.1
openstack user create --domain default --password-prompt placement
openstack role add --project service --user placement admin
openstack service create --name placement --description "Placement API" placement
openstack endpoint create --region RegionOne placement public http://openstack01:8778
openstack endpoint create --region RegionOne placement internal http://openstack01:8778
openstack endpoint create --region RegionOne placement admin http://openstack01:8778

yum install openstack-nova-api openstack-nova-conductor \
  openstack-nova-console openstack-nova-novncproxy \
  openstack-nova-scheduler openstack-nova-placement-api
===================
vi /etc/nova/nova.conf
[DEFAULT]
enabled_apis = osapi_compute,metadata
[api_database]
connection = mysql+pymysql://nova:NOVA_DBPASS@openstack01/nova_api
[database]
connection = mysql+pymysql://nova:NOVA_DBPASS@openstack01/nova
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@openstack01
[api]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = demo01
[DEFAULT]
my_ip = 10.0.0.11
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
[vnc]
enabled = true
vncserver_listen = $my_ip
vncserver_proxyclient_address = $my_ip
[glance]
api_servers = http://openstack01:9292
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://openstack01:35357/v3
username = placement
password = demo01
=========================
vi /etc/httpd/conf.d/00-nova-placement-api.conf
<Directory /usr/bin>
   <IfVersion >= 2.4>
      Require all granted
   </IfVersion>
   <IfVersion < 2.4>
      Order allow,deny
      Allow from all
   </IfVersion>
</Directory>
systemctl restart httpd
#에러가 좀 난다 무시하라고 한다.
su -s /bin/sh -c "nova-manage api_db sync" nova
su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova

su -s /bin/sh -c "nova-manage db sync" nova
nova-manage cell_v2 list_cells

systemctl enable openstack-nova-api.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service
systemctl start openstack-nova-api.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service
###########################compute openstack02
yum install centos-release-openstack-pike
yum install https://rdoproject.org/repos/rdo-release.rpm
yum upgrade
yum install python-openstackclient
yum install openstack-selinux
yum install openstack-nova-compute
yum install openstack-nova-compute
vi /etc/nova/nova.conf
[DEFAULT]
enabled_apis = osapi_compute,metadata
[api]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = demo01
[DEFAULT]
my_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@openstack01
[DEFAULT]
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
[vnc]
enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = $my_ip
novncproxy_base_url = http://openstack01:6080/vnc_auto.html

[glance]
api_servers = http://openstack01:9292

[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://openstack01:35357/v3
username = placement
password = demo01

egrep -c '(vmx|svm)' /proc/cpuinfo
[libvirt]
virt_type = qemu

systemctl enable libvirtd.service openstack-nova-compute.service
systemctl start libvirtd.service openstack-nova-compute.service

###########################controller openstack01

openstack compute service list --service nova-compute
su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
=============
뉴트론 설치
=============
###########################openstack01,02
vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=INTERFACE_NAME
TYPE=Ethernet
ONBOOT="yes"
BOOTPROTO="none"
###########################openstack01
mysql -u root -p
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' \
  IDENTIFIED BY 'NEUTRON_DBPASS';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' \
  IDENTIFIED BY 'NEUTRON_DBPASS';
flush privileges;
openstack user create --domain default --password-prompt neutron
openstack role add --project service --user neutron admin
openstack service create --name neutron \
  --description "OpenStack Networking" network
openstack endpoint create --region RegionOne \
  network public http://openstack01:9696
openstack endpoint create --region RegionOne \
  network internal http://openstack01:9696
openstack endpoint create --region RegionOne \
  network admin http://openstack01:9696
===================self service network
yum install openstack-neutron openstack-neutron-ml2 \
  openstack-neutron-linuxbridge ebtables
vi /etc/neutron/neutron.conf
[database]
connection = mysql+pymysql://neutron:NEUTRON_DBPASS@openstack01/neutron
[DEFAULT]
core_plugin = ml2
service_plugins = router
allow_overlapping_ips = true
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@openstack01
[DEFAULT]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = demo01
[DEFAULT]
notify_nova_on_port_status_changes = true
notify_nova_on_port_data_changes = true
[nova]
auth_url = http://openstack01:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password = demo01

[oslo_concurrency]
lock_path = /var/lib/neutron/tmp

vi /etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider

[ml2_type_vxlan]
vni_ranges = 1:1000
[securitygroup]
enable_ipset = true
vi /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[linux_bridge]
physical_interface_mappings = provider:PROVIDER_INTERFACE_NAME

[vxlan]
enable_vxlan = true
local_ip = OVERLAY_INTERFACE_IP_ADDRESS
l2_population = true

[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

vi /etc/neutron/l3_agent.ini

[DEFAULT]
interface_driver = linuxbridge

vi /etc/neutron/dhcp_agent.ini

[DEFAULT]
interface_driver = linuxbridge
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = true
===============
vi /etc/neutron/metadata_agent.ini

[DEFAULT]
nova_metadata_host = openstack01
metadata_proxy_shared_secret = METADATA_SECRET
vi /etc/nova/nova.conf
[neutron]
url = http://openstack01:9696
auth_url = http://openstack01:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = demo01
service_metadata_proxy = true
metadata_proxy_shared_secret = METADATA_SECRET

ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini
su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \
  --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
systemctl restart openstack-nova-api.service
systemctl enable neutron-server.service \
neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
neutron-metadata-agent.service
systemctl start neutron-server.service \
neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
neutron-metadata-agent.service
systemctl enable neutron-l3-agent.service
systemctl start neutron-l3-agent.service
###########################compute openstack02

yum upgrade -y
yum install centos-release-openstack-pike
yum install https://rdoproject.org/repos/rdo-release.rpm
yum upgrade
yum install python-openstackclient
yum install openstack-selinux

yum install openstack-neutron-linuxbridge ebtables ipset

vi /etc/neutron/neutron.conf
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@openstack01
[DEFAULT]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = neutron
password = demo01
[oslo_concurrency]
lock_path = /var/lib/neutron/tmp
vi /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[linux_bridge]
physical_interface_mappings = provider:eth0
[vxlan]
enable_vxlan = true
local_ip = 10.0.0.45
l2_population = true
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

vi /etc/nova/nova.conf
[neutron]
url = http://openstack01:9696
auth_url = http://openstack01:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = demo01
systemctl restart openstack-nova-compute.service
systemctl enable neutron-linuxbridge-agent.service
systemctl start neutron-linuxbridge-agent.service

=============
호라이즌 설치
=============
###########################controller openstack01
yum install openstack-dashboard
vi  /etc/openstack-dashboard/local_settings
OPENSTACK_HOST = "openstack01"
ALLOWED_HOSTS = ['*']
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
CACHES = {
    'default': {
         'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
         'LOCATION': 'openstack01:11211',
    }
}
OPENSTACK_HOST="10.0.0.44"
OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_API_VERSIONS = {
    "identity": 3,
    "image": 2,
    "volume": 2,
}
OPENSTACK_KEYSTONE_DEFAULT_DOMAIN = "Default"
OPENSTACK_KEYSTONE_DEFAULT_ROLE = "user"
OPENSTACK_NEUTRON_NETWORK = {
    ...
    'enable_router': False,
    'enable_quotas': False,
    'enable_distributed_router': False,
    'enable_ha_router': False,
    'enable_lb': False,
    'enable_firewall': False,
    'enable_vpn': False,
    'enable_fip_topology_check': False,
}
TIME_ZONE = "Asia/Seoul"
systemctl restart httpd.service memcached.service
###########################Block Storage openstack03
yum upgrade -y
yum install centos-release-openstack-pike
yum install https://rdoproject.org/repos/rdo-release.rpm
yum upgrade
yum install python-openstackclient
yum install openstack-selinux
yum install lvm2 device-mapper-persistent-data
systemctl enable lvm2-lvmetad.service
systemctl start lvm2-lvmetad.service
fdisk -l
pvcreate /dev/sdc
vi /etc/lvm/lvm.conf
devices {
filter = [ "a/sdc/","r/.*/" ]
}

yum install openstack-cinder targetcli python-keystone

vi /etc/cinder/cinder.conf

[database]
connection = mysql+pymysql://cinder:CINDER_DBPASS@openstack01/cinder
[DEFAULT]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = demo01
[DEFAULT]
my_ip = 10.0.0.44

[lvm]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-volumes
iscsi_protocol = iscsi
iscsi_helper = lioadm
[DEFAULT]
enabled_backends = lvm

[DEFAULT]
glance_api_servers = http://openstack01:9292

[oslo_concurrency]
lock_path = /var/lib/cinder/tmp

systemctl enable openstack-cinder-volume.service target.service
systemctl start openstack-cinder-volume.service target.service

########################controller openstack01
mysql -u root -p
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' \
  IDENTIFIED BY 'CINDER_DBPASS';
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' \
  IDENTIFIED BY 'CINDER_DBPASS';
flush privileges;
exit

openstack user create --domain default --password-prompt cinder
openstack role add --project service --user cinder admin
openstack service create --name cinderv2 \
  --description "OpenStack Block Storage" volumev2
openstack service create --name cinderv3 \
  --description "OpenStack Block Storage" volumev3
openstack endpoint create --region RegionOne \
  volumev2 public http://openstack01:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev2 internal http://openstack01:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev3 admin http://openstack01:8776/v3/%\(project_id\)s
yum install openstack-cinder
vi /etc/cinder/cinder.conf
[database]
connection = mysql+pymysql://cinder:CINDER_DBPASS@openstack01/cinder

[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@openstack01
[DEFAULT]
auth_strategy = keystone
[keystone_authtoken]
auth_uri = http://openstack01:5000
auth_url = http://openstack01:35357
memcached_servers = openstack01:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = demo01

[DEFAULT]
my_ip = 10.0.0.44
[oslo_concurrency]
lock_path = /var/lib/cinder/tmp
su -s /bin/sh -c "cinder-manage db sync" cinder
vi /etc/nova/nova.conf

[cinder]
os_region_name = RegionOne
yum install openstack-cinder
vi /etc/cinder/cinder.conf
[DEFAULT]
backup_driver = cinder.backup.drivers.swift
backup_swift_url = SWIFT_URL
openstack catalog show object-store
systemctl enable openstack-cinder-backup.service
systemctl start openstack-cinder-backup.service
####################contoller openstack01
openstack volume service list

댓글

댓글 쓰기

이 블로그의 인기 게시물

LSF (GPU 스케쥴링) 명령어 사용법

=================주로사용하는 명령어============================ # 큐 확인 bqueues # 큐 선택방법 Deep learning을 할거면 s822lc_p100nvme 라는 queue job을 submit HPC을 할거면 s822lc_p100 라는 queue job을 submit # 스케쥴링 걸어서 실행 (-x는 혼자 한장의 GPU만 쓰더라도 전체를 독점적으로 쓰겠다는 옵션) bsub -R "select[ngpus>0] rusage[ngpus_excl_p=1]" -x -q s822lc_p100nvme PYTHONPATH=/~/bin/python /~/cifar10_main.py # 수행 중인 Job-id에 대한 정보 (Job-id는 위에 스케쥴링 걸어서 실행하면 보여줌) bhist -l Job-id =================명령어 설명============================ bqueues : job을 submit할 큐의 정보를 보여줍니다 bsub : job을 큐에 submit 해줍니다 bjobs : 큐에 submit된 job의 상태를 보여줍니다 bhist : 현재 수행 중인, 혹은 이미 수행이 끝난 job의 history를 보여줍니다 bkill : submit되어 현재 수행 중인 상태의 job을 도중에 kill 시켜 줍니다 bhosts : 수퍼컴 클러스터 내의 노드들 상황을 보여줍니다. ==================예제=========================== # 쉘스크립트 작성 b7p193aa@p10login1:~$ cat cifar10.sh  #!/bin/bash  source /opt/DL/tensorflow/bin/tensorflow-activate  source /opt/DL/bazel/bin/bazel-activate  export FLOWERS_DIR=/gpfs/gpfs_gl4_16mb/...
자세한 내용 보기

CentOS 7 리부팅 없이 새 디스크 인식 (find a new disk without reboot)

가상화 환경 혹은 클라우드에서 새로운 디스크를 추가하고 리눅스 서버 리부팅 없이 인식하여 사용할 필요성이 생겼다.  =============================== cd /sys/class/scsi_host ls -al host0 부터 들어가서 아래와 같이 한다 echo "- - -" > /sys/class/scsi_host/host0/scan host1이 있으면 그것도.. echo "- - -" > /sys/class/scsi_host/host1/scan fisk -l 하면 나온다
자세한 내용 보기

python에서 hive 사용

OS에 sasl 설치 yum install libsasl2-dev 파이선 패키지 설치 pip install sasl pip install thrift pip install thrift-sasl pip install PyHive 연결 from pyhive import hive conn = hive.Connection(host="hive server ip", port=port, username="ID") 사용 cursor = conn.cursor() cursor.execute("SELECT 칼럼 FROM 테이블") for result in cursor.fetchall():   use_result(result) or import pandas as pd df = pd.read_sql("SELECT cool_stuff FROM hive_table", conn) ============================= pip install sasl pip install thrift pip install thrift-sasl pip install pyhs2 import pyhs2 with pyhs2.connect(host='localhost',                    port=10000,                    authMechanism="PLAIN",                    user='root',                    password='test',         ...
자세한 내용 보기