2 years ago
#59031
Nitish Pandey
Problem while creating the NLP model using tensorflow?
I have created a spam classifier model using tensorflow.. ( tokenization , embedding layer and lstm ).
model=Sequential()
model.add(Embedding(voc_size,32,input_length=sent_len))
model.add(LSTM(units=128))
model.add(Dropout(0.5))
model.add(BatchNormalization())
model.add(Dense(units=1,activation='sigmoid'))
I have trained the model with my data and got an accuracy of about 97-98%.
I processed my data using the below function.
def process(data):
data=re.sub('[^a-zA-Z]'," ",data)
data=data.lower()
data=data.split()
data=[lemmatizer.lemmatize(word) for word in data if word not in set(stopword)]
data=" ".join(data)
return data
I have predicted my new custom input as :
def predict(data):
data=[process(sent) for sent in data]
data=tokenizer.texts_to_sequences(data)
data=pad_sequences(data,maxlen=sent_len,padding='pre')
pred=(mymodel.predict(data)>0.5).astype('int32').tolist()
return pred
data=input()
predict(data)
But I am receiving the msg as non-spam for every msg I input..no matter what I enter.Look here
I don't know what is happening. I am not able to deploy such a model.
python
tensorflow
nlp
tokenize
word-embedding
0 Answers
Your Answer