本文主要是介绍【强化学习】使用LSTM模型来生成歌词，对大家解决编程问题具有一定的参考价值，需要的程序猿们随着小编来一起学习吧！

目录

问题描述：

解决思路：

1.LSTM算法

2.具体实现

实现步骤

代码展示

完成截图

参考：

问题描述：

        选择一位歌手的英文歌曲，以txt文件存储在python文件同级。

参考歌词文件：

Hands-On-Reinforcement-Learning-with-Python/ZaynLyrics.txt at master · PacktPublishing/Hands-On-Reinforcement-Learning-with-Python · GitHub

解决思路：

1.LSTM算法

        LSTM算法建立在RNN算法的基础上。LSTM解决了，RNN算法在训练过程中会出现梯度消失的情况。

        在LSTM单位中，有三种门槛。分别是忘记门槛，输入门槛和输出门槛。数据的更新将是沿着从左到右来进行更新。

其中三种门槛的作用分别是

        a)选择进行更新的是哪种数据

        b)选择哪种数据应该存入到内存中

        c)决定该输出哪种数据

2.具体实现

实现步骤

a)导入数据包和文件读取：

import numpy as np
import tensorflow as tf

with open("ZaynLyrics.txt","r") as f:
    data = f.read()
    data = data.replace('\n','')
    data = data.lower()

# 检验数据是否导入成功
# print(data[:50])

b)然后，将所有字符存储在all_chars变量中:

#store all the charachters in the all_chars variable
all_chars = list(set(data))

c)将唯一字符的数量存储在unique_chars中:

#store the number of unique characters 
unique_chars = len(all_chars)

d)我们还将字符总数存储在total_chars中:

#store the total number of character
total_chars =len(data)

e)建立字符与索引相互之间的映射。如：char_to_ix 代表字符到索引的映射

char_to_ix = {ch:i for i ,ch in enumerate(all_chars)}
ix_to_char = {i:ch for i ,ch in enumerate(all_chars)}

f）接下来，定义一个generate_batch函数，它将生成输入值和目标值。目标值就是i乘以输入值的位移

def generate_batch(seq_length,i):
    inputs = [char_to_ix[ch] for ch in data[i:i+seq_length]]
    targets = [char_to_ix[ch] for ch in data[i+1:i+seq_length +1]]
    inputs = np.array(inputs).reshape(seq_length,1)
    targets = np.array(targets).reshape(seq_length,1)
    return inputs,targets

g)定义序列长度、学习率和节点数

seq_length = 25
learning_rate = 0.1
num_nodes = 300

h)使用tensorflow提供的BasicLSTMCell()函数来构建LSTM单元

def build_rnn(x):
    cell = tf.compat.v1.nn.rnn_cell.BasicLSTMCell(num_units=num_nodes,activation =tf.nn.relu)
    outputs,states = tf.compat.v1.nn.dynamic_rnn(cell,x,dtype=tf.float32)
    return outputs,states

i)为x,y创建内存和数据类型变换,同时创建onehot表示x和y

tf.compat.v1.disable_eager_execution()

X = tf.compat.v1.placeholder(tf.float32,[None,1])
Y = tf.compat.v1.placeholder(tf.float32,[None,1])

X = tf.cast(X,tf.int32)
Y = tf.cast(Y,tf.int32)

X_onehot = tf.one_hot(X,unique_chars)
Y_onehot = tf.one_hot(Y,unique_chars)

j)调用函数来得到输出，以及对输出做转置

outputs,states = build_rnn(X_onehot)

outputs=tf.transpose(outputs,perm=[1,0,2])

k)初始化权重和偏差，并与输出相乘

#initialize the weights and biases
W = tf.Variable(tf.random.normal((num_nodes,unique_chars),stddev = 0.001))
B = tf.Variable(tf.zeros((1,unique_chars)))


#calculate output by multiplying the ouput with weights and add bias
Ys = tf.matmul(outputs[0],W)+B

l)执行激活函数，获得概率以及计算损失函数

#perform softmax activation and get the probabilities
prediction = tf.nn.softmax(Ys)

#calculate the cross_entropy loss
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels = Y_onehot,logits=Ys))

m)编辑编译器和对模型进行预测

# optimiser = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)
optimiser = tf.compat.v1.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)
def predict(seed,i):
    x = np.zeros((1,1))
    x[0][0] =seed
    indices = []
    for t in range(i):
        p = sess.run(prediction,{X:x})
        index = np.random.choice(range(unique_chars),p=p.ravel())
        x[0][0] =index
        indices.append(index)
    return indices

n)设置相关参数，开始建立模型

batch_size = 100
total_batch = int(total_chars // batch_size)
epoch = 1000
shift = 0

init = tf.compat.v1.global_variables_initializer()

with tf.compat.v1.Session() as sess:
    
    sess.run(init)
    for epoch in range(epoch):
        print("epoch:{}".format(epoch))
        if shift +batch_size +1 >= len(data):
            shift = 0

        for i in range(total_batch):
            inputs,targets = generate_batch(batch_size,shift)
            shift += batch_size

            #calucalate loss
            if (i %100 ==0):
                loos =sess.run(cross_entropy,feed_dict = {X:inputs,Y:targets})

                index = predict(inputs[0],200)

                txt = ''.join(ix_to_char[ix] for ix in index)
                print('iteration: %i'%(i))
                print('\n%s\n'%(txt,))

            sess.run(optimiser,feed_dict ={X:inputs,Y:targets})

代码展示

import numpy as np
import tensorflow as tf

with open("ZaynLyrics.txt","r") as f:
    data = f.read()
    data = data.replace('\n','')
    data = data.lower()

# print(data[:50])

#store all the charachters in the all_chars variable
all_chars = list(set(data))

#store the number of unique characters 
unique_chars = len(all_chars)

#store the total number of character
total_chars =len(data)

#create a mapping between each character to their index
char_to_ix = {ch:i for i ,ch in enumerate(all_chars)}
ix_to_char = {i:ch for i ,ch in enumerate(all_chars)}


#define a generate_batch function ,generate input and target values
def generate_batch(seq_length,i):
    inputs = [char_to_ix[ch] for ch in data[i:i+seq_length]]
    targets = [char_to_ix[ch] for ch in data[i+1:i+seq_length +1]]
    inputs = np.array(inputs).reshape(seq_length,1)
    targets = np.array(targets).reshape(seq_length,1)
    return inputs,targets


seq_length = 25
learning_rate = 0.1
num_nodes = 300

def build_rnn(x):
    cell = tf.compat.v1.nn.rnn_cell.BasicLSTMCell(num_units=num_nodes,activation =tf.nn.relu)
    #cell = tf.contrib.rnn.BasicLSTMCell(num_units=num_nodes,activation =tf.nn.relu)
    outputs,states = tf.compat.v1.nn.dynamic_rnn(cell,x,dtype=tf.float32)
    return outputs,states


tf.compat.v1.disable_eager_execution()


X = tf.compat.v1.placeholder(tf.float32,[None,1])
Y = tf.compat.v1.placeholder(tf.float32,[None,1])

X = tf.cast(X,tf.int32)
Y = tf.cast(Y,tf.int32)


X_onehot = tf.one_hot(X,unique_chars)
Y_onehot = tf.one_hot(Y,unique_chars)

# outputs,states = build_rnn(X_onehot)
outputs,states = build_rnn(X_onehot)

outputs=tf.transpose(outputs,perm=[1,0,2])

#initialize the weights and biases
W = tf.Variable(tf.random.normal((num_nodes,unique_chars),stddev = 0.001))
B = tf.Variable(tf.zeros((1,unique_chars)))


#calculate output by multiplying the ouput with weights and add bias
Ys = tf.matmul(outputs[0],W)+B

#perform softmax activation and get the probabilities
prediction = tf.nn.softmax(Ys)

#calculate the cross_entropy loss
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels = Y_onehot,logits=Ys))

# optimiser = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)
optimiser = tf.compat.v1.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy)
def predict(seed,i):
    x = np.zeros((1,1))
    x[0][0] =seed
    indices = []
    for t in range(i):
        p = sess.run(prediction,{X:x})
        index = np.random.choice(range(unique_chars),p=p.ravel())
        x[0][0] =index
        indices.append(index)
    return indices

batch_size = 100
total_batch = int(total_chars // batch_size)
epoch = 1000
shift = 0

init = tf.compat.v1.global_variables_initializer()

with tf.compat.v1.Session() as sess:
    
    sess.run(init)
    for epoch in range(epoch):
        print("epoch:{}".format(epoch))
        if shift +batch_size +1 >= len(data):
            shift = 0

        for i in range(total_batch):
            inputs,targets = generate_batch(batch_size,shift)
            shift += batch_size

            #calucalate loss
            if (i %100 ==0):
                loos =sess.run(cross_entropy,feed_dict = {X:inputs,Y:targets})

                index = predict(inputs[0],200)

                txt = ''.join(ix_to_char[ix] for ix in index)
                print('iteration: %i'%(i))
                print('\n%s\n'%(txt,))

            sess.run(optimiser,feed_dict ={X:inputs,Y:targets})

完成截图

结果分析：

        从图中标红可以看出，随着一次次迭代。模型已经可以从一开始的无语法状态，迭代出基本的英语单词。但是迭代时间过长，需要增加模型的精度和效率

参考：

《Hands-on Reinforcement Learning with Python. Master Reinforcement and Deep Reinforcement Learning using OpenAI Gym and TensorFlow》

这篇关于【强化学习】使用LSTM模型来生成歌词的文章就介绍到这儿，希望我们推荐的文章对大家有所帮助，也希望大家多多支持为之网！