tokenizer = AutoTokenizer.from_pretrained('roberta-base') I never faced this issue before and it was working absolutely fine earlier. I am clueless.
3 from transformers import AutoTokenizer, AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="cuda:3", ) There are many GPUs on the server, but I can only use two of them. How should I configure device_map (or other parameters) so that the model runs on both GPUs?
from transformers import AutoTokenizer path_autotokenizer = ... new_tokenizer = AutoTokenizer.from_pretrained(path_autotokenizer) and it doesn't work with transformers==4.41.2 - can't load at all. With dependencies below, it does work:
AutoTokenizer.from_pretrained fails if the specified path does not contain the model configuration files, which are required solely for the tokenizer class instantiation. In the context of run_language_modeling.py the usage of AutoTokenizer is buggy (or at least leaky). There is no point to specify the (optional) tokenizer_name parameter if it's identical to the model name or path. Therefore ...
from transformers import AutoTokenizer, AutoConfig tokenizer = AutoTokenizer.from_pretrained('distilroberta-base') config = AutoConfig.from_pretrained('distilroberta-base') tokenizer.save_pretrained('YOURPATH') config.save_pretrained('YOURPATH') tokenizer = AutoTokenizer.from_pretrained('YOURPATH') I recommend to either use a different path for the tokenizers and the model or to keep the ...
tokenizer = AutoTokenizer.from_pretrained(path, local_files_only=True) The path (models/yu) leads to a directory which contains the files from above (config.json, pytorch_model.bin, training_args.bin), it is relative to the location of my .ipynb file. Yet when I try to load it I get the following error: OSError: Can't load tokenizer for 'models ...
The simplest way to let AutoTokenizer load .from_pretrained is to follow the answer that @cronoik posted in the comment, using PreTrainedTokenizerFast, i.e. adding a few lines to your existing code:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("google/ul2") model = AutoModelForSeq2SeqLM.from_pretrained("google/ul2") I get an out of memory error, as the model only seems to be able to load on a single GPU. However, while the whole model cannot fit into a single 24GB GPU card, I have 6 of these and would like to know if there is a ...